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20  ABSTRACT  ( Continue  on  rovarao  oldo  it  nacoaaary  and  Idontlfy  by  block  numbar) 

The  Military  Message  Experiment  (MME)  is  designed  to  evaluate  the  utility  of  user-oriented 
message  processing  systems  in  a  military  environment  and  to  aid  in  determining  the  features  useful 
in  such  a  system.  The  experiment  is  a  cooperative  effort  between  the  Commander-in-Chief,  Pacific, 
the  Navy,  and  the  Defense  Advanced  Research  Projects  Agency.  To  conduct  the  experiment,  a 
PDP-10-based  system  has  been  installed  at  CINCPAC  Headquarters  for  use  by  a  portion  of  the 
Operations  Directorate.  The  message  processing  functionality  is  provided  by  SIGMA,  a  program 
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20.  Abstract  (Continued) 

written  by  the  Information  Sciences  Institute  of  the  University  of  Southern  California.  It  is  sup¬ 
ported  by  the  TENEX  operating  system,  and  the  user  terminals  are  modified  HP-2649A  CRTs.  -A - < 

-find  report, -to  be  pubHshed-ia-April  1980,  is  planned.  > 

The  MME  system  is  designed  to  give  the  user  the  capability  to  handle  his  message  traffic  (both 
incoming  and  outgoing,  formal  and  informal)  on  the  system.  The  system  enforces  multilevel  security 
rules  based  on  a  modification  of  the  security  kernel  model  developed  at  Mitre.  The  rule  enforcement 
in  not  rigorous  enough  for  certification,  but  it  is  sufficiently  rigorous  to  determine  the  effects  on  the 
users’  interactions  with  the  system.  Most  of  the  functions  needed  for  a  user’s  message-related  tasks 
are  provided  by  the  system:  message  distribution  and  redistribution,  ^electronic  readboard*  con¬ 
struction,  message  filing,  message  replies,  message  commenting  and  “chopping,”  and  message  release. 

*7 

Based  on  the  limited  amount  of  time  the  system  has  been  in  use,  the  following  tentative  con¬ 
clusions  have  been  drawn: 

a.  An  Automated  Message  Handling  System  is  useful  in  a  military  environment. 

b.  An  Automated  Message  Handling  System  must  be  an  integral  component  of  a  communications 
system  whose  overall  design  carefully  considers  the  ways  in  which  the  users  will  interact  with 
the  system.  The  total  system  must  be  reliable;  it  must  provide  services  to  everyone  involved 
with  message  handling  in  the  user  environment;  it  must  provide  a  proper  balance  between 
ephemeral  displays  and  paper  copies. 

c.  An  Automated  Message  Handling  System,  with  a  carefully  designed  user  interface,  can  provide 
a  useful  subset  of  functions  that  can  be  made  available  to  the  casual  user  without  the  need 
for  extensive  formal  training. 

The  objectives  of  the  experiment  are  being  met.  Many  of  the  answers  that  are  needed  by 
designers  of  future  Automated  Message  Handling  Systems  will  be  provided  before  the  conclusion 
of  the  experiment.  Multi-level  security  is  one  of  the  important  research  issues  that  is  being  addressed 
during  the  experiment.  A  section  of  the  final  report  will  be  devoted  to  a  description  of  the  security 
model  used,  the  system  design  issues  that  it  raised,  and  the  effect  it  had  on  the  users’  interactions 
with  the  MME  system. 


W>c  TAB 

^-announced 
Jaatifica ttaa 


A'al* and/or 
special 


■**.!  ,K 


DEPARTMENT  OF  THE  NAVY 

NAVAL  TELECOMMUNICATIONS  COMMAND 
4401  MASSACHUSETTS  AVENUE.  N.W. 
WASHINGTON.  D.C.  20390 


IN  REPLY  refer  to 


Ser  06/6357 

39  DEi  i97fi 


FIRST  ENDORSEMENT  on  Commander,  Naval  Electronic  Systems  Conmand  Itr  310:NMT:crh 
Ser:  570-310  of  2  November  1979 

From:  Commander,  Naval  Telecommunications  Command 
To:  Distribution  List 


Subj:  Military  Message  Experiment  (MME)  Mid-Experiment  Report 

1.  Forwarded. 

2.  Enclosure  (1)  is  the  second  of  three  Military  Message  Experiment  reports  to 
be  issued  and  covers  the  period  from  November  1978  through  March  1979.  The 
third  and  final  report  will  be  issued  during  April  1980, 

•G.  B.  SHICK,  JR 

Distribution: 

OUSDRE  (C3I)  Information  System  (LCol .  Wilcox)  (2) 

JCS  (J3-ISD)  (2) 

DCA  (Code  534)  (3) 

DARPA  (IPTO)  (3) 

CNO  (OP-94)  (3) 

CINCPAC  (J3)  (3) 

CINCPAC  (J6)  (3) 

Copy  to: 

AFCCPC  (Maj.  R.  Harris) 

AFIS/IND  (Mr.  W.  Lamour) 

CCTC  (Code  C634) 

CDRUSACC  (CC-OPA-WA  &  CC-OPS-T) 

CDR7THSIGC0MD  (CCN-PO-N) 

CINCAD 
CINCLANT 
CINCMAC 
CINCSAC 

CTEC  (Dr.  Bersoff) 

DA  (DACC-SIF)  (2) 

DCEC  (Code  B-800)  (2) 

DDC 

DIA  (RS0-3&RSP-2)  (2) 


3 


Ser  06/6357 

19  L  - 1979 

Dir.  DARPA  Regional  Office,  Europe 
Dir.  DARPA  Regional  Office,  Pacific 
HQ  SAC  (DXT) 

HQ  TRADOC  (ATCD-C-C) 

HQUSAF  (AF/XOKCR)  (3) 

Information  Sciences  Institute  (Mr.  K.  Uncapher) 

MITRE  Corp. 

National  Bureau  of  Standard  (Dr.  S.  Kimbleton) 

National  Science  Foundation  (Mr.  P.  Custer) 

NAVSEA  (Code  047) 

NAVTELCOM 

NAVELEXSYSCOM  (Code  310) 

NIPSSA  (NIC  00Q4,  Mr.  R.  Gray) 

NOSC  (Code  8122)  (2) 

NRL  (Code  7503) 

ONR  (Code  100) 

PTAO  (T.  Young) 

RADC  (Dr.  K.  Plante) 

SHAPE  (Mr.  W.  Stoney) 

USCINCEUR 
USCINCRED 
USREDCOM  (RCJ6-T) 

USCINCSO 

WSEO/DCA  (Col.  Pixton,  Maj.  Noll)  (2) 


4 


DEPARTMENT  OF  THE  NAVY 
NAVAL  ELECTRONIC  SYSTEMS  COMMANO 
WASHINGTON.  O.C.  20360 


in  imply  ncrCR  TO 

310:NMT:crh 
Ser:  570-310 
2  November  1979 


From:  Commander,  Naval  Electronic  Systems  Command 

To:  Distribution 

Via:  Commander,  Naval  Telecommunications  Command 

Subj:  Military  Message  Experiment  (MME)  Mid-Experiment  Report 

Ref:  (a)  MME  Memorandum  of  Agreement  September  1978 

Enel:  (1)  MME  Mid-Experiment  Report 

1.  In  accordance  with  reference  (a),  the  subject  report  is  submitted  as 
enclosure  (1).  The  report  covers  the  period  of  the  Experiment  from 
November  1C7R  to  Mar»ah  1Q7Q  and  includes  the  last  three  months  of  Limited 
Experimental  Use  (LEU),  the  first  month  of  Full  Experimental  Use  (FEU),  and 
the  Command  Post  Exercise  POWER  PLAY  79.  Preliminary  Conclusions  are  provided 
in  Section  6,  while  CINCPAC's  User  Perceptions  are  included  as  Appendix  A. 


2.  Additional  copies  of  this  report  may  oe  requested  from  Code  7503,  Naval 
Research  Laboratory,  Washington,  D.C.  20375. 


Roger  L.  Reasonover,  Jr. f 
By  direction 


Distribution: 

OUSDRE  (C3l)  Information  System  (LCol.  Wilcox) (2) 
JCS  ( J3-ISD)  (2) 

DCA  (Code  534)  (3) 

DARPA  ( IPTO)  (2) 

CNO  (OP -94)  (3) 

C IN CP AC  (J3)  (3) 

CINCPAC  (J6)  (3) 


Subj : 


Military  Message  Experiment  (MME)  Mid-Experiment  Report 


Copy  to: 

AFCCPC  (Maj.  R.  Harris) 

AFIS/IND  (Mr.  K.  Lamour) 

CCTC  (Code  C634) 

CDRUSACC  (CC-OPA-WA  &  CC-OPS-T) 

CDR7THSIQC0MD  (CCN-PO-N) 

CINCAD 

CINCLANT 

CINCMAC 

CINCSAC 

CTEC  (Dr.  Bersoff) 

DA  (DACC-SIF)  (2) 

DCEC  (Code  B-800)  (2) 

DDC 

DIA  (RS0-3&RSP-2)  (2) 

Dir.  DARPA  Regional  Office,  Europe 
Dir.  DARPA  Regional  Office,  Pacific 
HQ  SAC  (DXT) 

HQ  TRADOC  (ATCD-C-C) 

HQUSAF  (AF/XOKCR)  (3) 

Information  Sciences  Institute  (Mr.  K.  Uncapher) 
MITRE  Corp. 

National  Bureau  of  Standard  (Dr.  S.  Kimbleton) 
National  Science  Foundation  (Mr.  P.  Custer) 
NAVSEA  (Code  047) 

NAV  TEL COM 

NAVELEXSYSCOM  (Code  310) 

NIPSSA  (NIC  00Q4,  Mr.  R.  Gray) 

NOSC  (Code  8122)  (2) 

NRL  (Code  7503) 

ONR  (Code  100) 

PTAO  (T.  Young) 

RADC  (Dr.  K.  Plante) 

SHAPE  (Mr.  W.  Stoney) 

USCINCEUR 
USCINCRED 
USREDC0M  (RCJ6-T) 

USCINCS0 

WSE0/DCA  (Col.  Pixton,  Maj.  Noll)  (2) 


CONTENTS 


1  EXECUTIVE  SUMMARY  .  1 

2  INTRODUCTION  .  5 

3  PATTERNS  OF  USE  .  7 

3.1  Amount  of  Use  .  9 

3.1.1  On-Line  Time  .  9 

3.1.2  System  Use .  10 

3.2  Type  of  Use . . .  12 

3.2.1  Function  Utility  .  12 

3.2.2  Styles  of  Use  .  13 

3.2.3  Flagged  Instructions  . .  14 

4  EXERCISE  PARTICIPATION  .  15 

4.1  Background .  15 

4.2  Pre-Exercise  Set-up  and  Training .  15 

4.3  MME  Participation  .  16 

4.3.1  Personnel .  17 

4.3.2  Highlights  .  18 

4.3.3  Problems .  18 

4.4  MME  System  Effectiveness .  19 

5  INTERFACE  TO  AUTODIN  .  2C 

5.1  Summary  of  Autodin  Traffic  .  20 

5.1.1  Incoming  Messages  .  20 

5.1.2  Outgoing  Messages  .  . .  20 

5.2  MME/LDMX  Interface  .  21 

5.2.1  Incoming  Messages  .  21 

5.2.2  Outgoing  Messages  .  21 

5.2.3  Retransmittals .  22 

5.2.4  Multisection  Messages .  22 

5.2.5  Readdressals  .  22 

5.2.6  Unclassified-EFTO  Messages  .  22 

5.2.7  Service  Messages  .  23 

5.2.8  Lack  of  Upward  Compatibility  in  LDMX  Releases  .  23 

5.2.9  Message  Formats .  23 

5.2.10  Duplicative  Uncoordinated  Control  Checks .  23 

6  CONCLUSIONS  .  24 

7  ACKNOWLEDGMENTS .  28 

8  REFERENCES  .  29 

APPENDIX  A .  30 

APPENDIX  B  .  43 


iii 


MILITARY  MESSAGE  EXPERIMENT 
MID  EXPERIMENT  REPORT 

SECTION  1 
EXECUTIVE  SUMMARY 


This  report  is  the  second  of  three  reports  to  be  prepared  during  the 
Military  Message  Experiment  (MME);  the  first  report  [reference  1]  was 
published  April  30,  1979,  and  describes  the  initial  portion  of  the  experiment 
from  May  1977  to  November  1978.  In  addition,  it  describes  the  objectives  of 
the  experiment  and  the  system  being  used  to  conduct  the  experiment.  This 
report  covers  the  period  from  November  1978  through  March  1979. 

The  Military  Message  Experiment  is  an  effort  to  evaluate  the  utility  of 
computer-aided  message  handling  in  a  military  environment  and  to  aid  in  the 
determination  of  the  type  of  automation  needed  in  message  processing  within 
military  environments  such  as  that  at  CINCPAC.  In  order  to  establish  a 
framework  for  the  experiment,  a  Memorandum  of  Agreement  [reference  2]  was 
signed  by  the  Director  of  the  Defense  Advanced  Research  Projects  Agency;  the 
Commander,  Naval  Telecommunications  Command;  the  Commander,  Naval  Electronic 
Systems  Command;  and  the  Commander-in-Chief,  Pacific,  in  December  1975  and 
revised  in  September  1978. 

Briefly,  the  experiment  consists  of  (a)  the  installation  of  an  Automated 
issage  Handling  System  that  links  a  selected  portion  of  the  CINCPAC  staff  to 
the  AUTODIN  communications  system,  (b)  on-site  support  and  training,  and  (c) 
an  evaluation  team  made  up  of  personnel  from  CINCPAC,  NRL,  NAVSEA,  NAVELEX, 
MITRE,  and  CTEC.  The  automated  message  handling  is  provided  by  the  MME  system 
consisting  of  the  SIGMA  message  processing  program  (written  by  the  Information 
Sciences  Institute  of  the  University  of  Southern  California)  operating  under 
the  Bolt,  Beranek,  and  Newman-developed  TENEX  operating  system  on  a  PDP-10 
computer  (KL  processor  and  one  million  words  of  memory).  Modified  Hewlett- 
Packard  HP-264 9A  CRTs  are  used  as  terminals. 

The  initial  system  was  installed  in  May  1977.  A  period  of  shakedown, 
training,  and  hardware  and  software  improvements  followed,  and  the  J3  staff 
began  limited  experimental  use  (LEU)  of  the  system  in  July  1978  with  SIGMA 
version  2.0.  The  following  are  significant  milestones  from  the  start  of 
limited  experimental  use  through  the  period  covered  by  this  report. 

Limited  Experimental  Use  Jul  1978 

SIGMA  Release  2.1  Sep  1978 

Hardware  upgrade  (KL  processor,  1M  words  memory)  Oct  1978 


Note:  Manuscript  submitted  August  7,  1979. 


SIGMA  Release  2.2 


Jan  1979 


Full  Experimental  Use  Feb  1979 

Exercise  Power  Play  79  Mar  1979 

Training  of  the  users  continued  through  the  limited  experimental  use 
phase,  and  the  system  responsiveness  and  functionality  were  upgraded  by  new 
hardware  and  by  software  releases  2.1  and  2.2.  The  system  was  also  used  to 
process  incoming  traffic  during  this  period. 

Full  experimental  use  of  the  system  began  in  February  1979  and  continued 
through  the  time  covered  by  this  report.  The  system  was  used  to  create, 
coordinate,  and  transmit  some  of  the  J3  staff's  outgoing  traffic  and  to 
construct  readboards  containing  annotated  messages  for  the  J3  staff. 

In  March,  participants  in  Exercise  Power  Play  79  used  the  system  as  a 
secondary  message  handling  device,  operating  in  parallel  with  the  current 
paper  system.  Routine  administrative  use  of  the  system  was  continued  during 
the  exercise.  No  significant  difference  was  noted  in  the  style  of  use  from 
that  found  in  day-to-day  operations,  with  the  exception  that  the  outgoing 
capabilities  of  the  system  were  used  more  heavily  by  the  exercise  participants. 
During  the  period  of  the  exercise,  however,  some  problems  with  the  software 
and  with  the  procedures  for  using  the  system  in  an  exercise  were  discovered. 

A  replay  of  the  exercise  has  been  scheduled  so  that  the  use  of  automated 
message  aids  during  a  crisis  can  be  better  evaluated. 

During  the  time  covered  by  this  report  there  have  been  some  periods  in 
which  the  system  has  been  very  unreliable,  and  these  periods  of  unreliability 
have  had  an  effect  on  the  user's  view  of  the  system.  In  December,  a  number  of 
problems  were  encountered  with  the  disk  memory  subsystem.  The  disk  drive  and 
the  backup  disk  controller  were  down  for  an  extended  period  and  user  files 
were  damaged.  The  overall  system  availability  for  December  was  86.8%  compared 
with  98.4%  and  97.4%  for  October  and  November.  The  disk  problems  continued 
intermittently  through  February.  In  early  March,  a  malfunctioning  filter  on 
the  main  power  line  was  removed.  This,  along  with  corrective  measures  to  the 
electromechanical  portions  of  the  disk  subsystem,  restored  the  MME  system 
stability.  The  system  availability  increased  from  92.9%  in  January  and  83%  in 
February  to  96.8%  in  March.  A  new  disk  subsystem  was  installed  in  April;  its 
use  started  in  the  period  immediately  after  that  covered  by  this  report.  The 
new  disk  is  more  reliable  and  doubles  the  amount  of  available  storage. 

In  general,  the  level  of  use  of  the  system  has  increased  for  ail  types  of 
users.  There  was  a  peak  during  the  exercise,  and  the  data  indicate  that  the 
amount  of  use  of  the  system  after  the  exercise  was  beginning  to  stabilize. 

A  number  of  problems  have  been  identified  in  the  MME  system’s  interface  to 
the  AUTODIN  via  the  LDMX.  While  some  of  these  problems  have  provided  useful 
information  to  the  designers  of  future  automated  message  handling  systems, 


others  have  prevented  the  examination  of  some  aspects  of  message  handling 
during  the  experiment.  In  particular: 

1.  Because  of  current  LDMX  operating  procedures  and  software  design,  TOP 
SECRET  messages  cannot  be  delivered  to  the  MME  system. 

2.  If  a  message  is  not  originally  distributed  to  J3  by  the  LDMX,  the 
LDMX  will  not  subsequently  deliver  the  message  to  the  MME  system  even 
if  requested  to  do  so  by  personnel  from  another  directorate. 

3.  Some  AUTODIN  format  checks  are  done  in  the  MME  system;  some  are  done 
in  the  LDMX.  The  MME,  on  occasion,  has  transmitted  an  improperly 
formatted  message  that  caused  the  LDMX  system  to  crash.  In  addition, 
a  message  that  has  been  transmitted  to  the  LDMX  by  the  MME  system  can 
no  longer  be  edited  in  the  MME  system  even  though  it  was  rejected  by 
the  LDMX. 

4.  There  are  incompatibilities  in  the  handling  of  nr  tisection  messages 
and  readdressals  by  the  two  systems. 

No  one  of  these  problems  is  disastrous,  but  they  indicate  the  need  for  the 
integrated  design  of  the  total  communications  system.  In  addition  to  the 
obvious  problems  that  are  listed,  there  are  subtle  differences  in  the  handling 
of  messges  in  the  MME  and  the  rest  of  the  communications  system  that  cause  the 
user  annoyance  (e.g.,  the  maintenance  of  the  plain  language  address  directory 
and  the  lack  of  rigorous  format  control  throughout  all  of  the  communications 
system  for  subject  line,  references,  pass-to  instructions,  and  keywords). 

User  observations  and  opinions  were  collected  by  using  a  detailed 
questionnaire  completed  by  both  daily  and  exercise  users  and  by  conducting 
interviews  with  senior  J3  personnel.  The  users  encountered  many  problems  with 
the  lack  of  reliability  and  responsiveness  of  the  system.  But  even  with  these 
problems,  the  users  believe  that  many  of  the  AMH  system  features  are  valuable 
and  assigned  an  overall  positive  rating  to  the  system.  Their  recommendations 
for  the  development  of  an  AMH  system  emphasize  that  the  problems  with  the  MME 
system  (such  as  slow  response  and  an  inadequate  number  of  terminals)  must  be 
corrected  in  future  systems.  They  also  emphasize  the  need  for  an  integrated 
design  of  the  total  message  handling  system.  The  results  of  the  questionnaire 
and  the  interviews,  the  documented  user  concerns  and  requests  for  changes,  and 
the  observations  of  the  on-site  evaluation  team  have  been  used  in  forming  the 
following  set  of  tentative  conclusions.  These  conclusions  are  discussed  in 
greater  detail  in  section  6  of  this  report. 

The  following  are  conclusions  concerning  the  MME  system  rather  than  the 
concept  of  Automated  Message  Handling  Systems. 

Conclusion  1.  The  lack  of  reliability  of  the  MME  system  has  been  a 
major  complaint  of  the  CINCPAC  staff  users  and  has  had  an  impact  on 
their  perception  of  the  MME  system  in  particular  and  automated 
message  handling  in  general. 


Conclusion  2.  Despite  system  problems,  the  users  appear  to  prefer 
the  MME  system  to  the  manual  message-handling  system.  The  users  were 
asked  to  indicate  their  preference  for  automated  or  manual  message 
handling  in  each  of  three  general  categories:  receiving  messages, 
filing  and  retrieving  messages,  and  creating  and  sending  messages. 

(See  Appendix  A  for  a  more  detailed  analysis.)  The  daily  users 
indicated  a  slight  preference  for  automated  message  handling  for 
receiving  messages  and  a  distinct  preference  for  automated  message 
handling  for  filing,  retrieving,  creating,  and  sending  messages. 

Within  the  general  category  of  receiving  messages,  the  users  indicated 
a  distinct  preference  for  the  manual  system  for  the  initial  review  of 
incoming  messages.  The  exercise  users  indicated  a  slight  preference 
for  the  manual  system  for  receiving  messages,  a  distinct  preference 
for  the  automated  system  for  message  filing  and  retrieval,  and  a 
slight  preference  for  the  automated  system  for  creating  and  sending 
messages. 

The  following  are  preliminary  conclusions  concerning  the  concept  of 
User-Oriented  Automated  Message  Handling  Systems. 

Conclusion  3.  An  Automated  Message  Handling  System  is  useful  in  a 
military  environment. 

Conclusion  4.  An  Automated  Message  Handling  System  must  be  an 
integral  component  of  a  communications  system  whose  overall  design 
carefully  considers  the  ways  in  which  the  users  will  interact  with 
the  system.  The  total  system  must  be  reliable;  it  must  provide 
services  to  everyone  involved  with  message  handling  in  the  user 
environment;  it  must  provide  a  proper  balance  between  ephemeral 
displays  and  paper  copies. 

Conclusion  5.  An  Automated  Message  Handling  System,  with  a  carefully 
designed  user  interface,  can  provide  a  useful  subset  of  functions 
that  can  be  made  available  to  the  casual  user  without  the  need  for 
extensive  formal  training. 

The  objectives  of  the  experiment  are  ''eing  met.  Many  of  the  answers  that 
are  needed  by  designers  of  future  Automated  Message  Handling  Systems  will  be 
provided  before  the  conclusion  of  the  experiment.  The  use  of  a  system  such  as 
the  MME  in  an  exercise  or  crisis  is  discussed  in  this  report;  the  exercise 
will  be  rerun  and  the  results  reported  in  the  final  report.  The  type  of  use 
that  such  a  system  gets  is  discussed  in  this  report  and  will  be  expanded  in 
the  final.  Interviews  will  be  conducted  and  further  analysis  of  the  data  will 
be  done  to  make  a  better  estimate  of  the  response  time  needed  by  users  in  a 
message  handling  system.  Multi-level  security  is  one  of  the  important 
research  issues  that  is  being  addressed  during  the  experiment.  A  section  of 
the  final  report  will  be  devoted  to  a  description  of  the  security  model  used, 
the  system  design  issues  that  it  raised,  and  the  effect  it  had  on  the  users’ 
interactions  with  the  MME  system. 


SECTION  2 


INTRODUCTION 


This  report  is  the  second  of  three  reports  to  be  prepared  dur r  ig  the 
military  message  experiment  (MME);  the  first  report  [reference  lj  was 
published  April  30,  1979  and  describes  the  initial  portion  of  the  experiment 
from  May  1977  to  November  1978.  In  addition,  it  describes  the  objectives  of 
the  experiment  and  the  system  being  used  to  conduct  the  experiment.  This 
report  covers  the  period  from  November  1978  through  March  1979. 

Briefly,  the  experiment  consists  of  (a)  the  installation  of  an  Automated 
Message  Handling  System  that  links  a  selected  portion  of  the  CINCPAC  staff  to 
the  AUTODIN  communications  system,  (b)  on-site  support  and  training,  and  (c) 
an  evaluation  team  made  up  of  personnel  from  CINCPAC,  NRL,  NAVSEA,  NAVELEX, 
MITRE,  and  CTEC. 

The  automated  message  handling  is  provided  by  the  MME  system  consisting  of 
the  SIGMA  message  processing  program  (written  by  the  Information  Sciences 
Institute  of  the  University  of  Southern  California)  operating  under  the  Bolt, 
Beranek,  and  Newman-developed  TENEX  operating  system  on  a  PDP-10  computer  (XL 
processor  and  one  million  words  of  memory).  Modified  Hewlett-Packard  HP-2649A 
CRTs  are  used  as  terminals. 

The  initial  system  was  installed  in  May  1977.  There  was  a  period  of 
shakedown,  training,  and  hardware  and  software  improvements.  The  J3  staff 
began  limited  experimental  use  (LEU)  of  the  system  in  July  1978  with  SIGMA 
version  2.0.  The  following  are  the  significant  milestones  from  the  start  of 
limited  experimental  use  through  the  period  covered  by  this  report. 


Limited  Experimental  Use  Jul  1978 
SIGMA  Release  2.1  Sep  1978 
Hardware  upgrade  (KL  processor,  1M  words  memory)  Oct  1978 
SIGMA  Release  2.2  Jan  1979 
Full  Experimental  Use  Feb  1979 
Exercise  Power  Play  79  Mar  1979 


Sigma  Release  2.1  upgraded  the  responsiveness  of  the  system.  Release  2.2  had 
changes  to  increase  performance  and  to  provide  additional  functionality;  it 
had  an  improved  facility  for  alerting  users  when  a  high-precedence  message 
arrives  and  an  improved  facility  for  coordinating  outgoing  messages.  At  the 
end  of  March,  there  were  20  terminals  installed  in  user  spaces.  In  designing 
the  experiment,  many  factors  were  weighed  in  determining  the  number  of  staff 
officers  to  participate  in  the  experiment  and  the  number  and  distribution  of 
terminals.  The  plan  to  use  a  total  of  25  terminals  recognized  that  the  number 
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was  coo  small  and  that,  ideally,  terminals  should  be  placed  so  Chat  each  staff 
officer  involved  with  messages  would  have  easy  access  to  a  terminal.  Given 
the  experiment  limitations,  the  ideal  distribution  of  terminals  was  not 
possible. 

During  the  time  covered  by  this  report  there  have  been  some  periods  in 
which  the  system  has  been  very  unreliable,  and  these  periods  of  unreliability 
have  had  an  effect  on  the  user's  view  of  the  system.  In  December,  a  number  of 
problems  were  encountered  with  the  disk  memory  subsystem.  The  disk  drive  and 
the  backup  disk  controller  were  down  for  an  extended  period,  and  user  files 
were  damaged.  The  overall  system  availability  for  December  was  86. 8%  compared 
with  98.4%  and  97.4%  for  October  and  November.  The  disk  problems  continued 
intermittently  through  February.  In  early  March,  a  malfunctioning  filter  on 
the  main  power  line  was  removed.  This,  along  with  corrective  measures  to  the 
electromechanical  portions  of  the  disk  subsystem,  restored  the  MME  system 
stability.  The  system  availability  increased  from  92.9%  in  January  and  83%  in 
February  to  96.8%  in  March.  A  new  disk  subsystem  was  installed  in  April;  its 
use  started  in  the  period  immediately  after  that  covered  by  this  report.  The 
new  disk  is  more  reliable  and  doubles  the  amount  of  available  storage. 

The  remainder  of  this  report  discusses  the  use  of  the  MME  System  from 
November  1978  through  March  1979.  The  observations  and  conclusions  in  this 
report  are  based  on  statistics  collected  and  analyzed  on-site  and  statistics 
that  were  automatically  collected,  written  on  tape,  and  shipped  to  Mitre, 
Bedford,  for  analysis.  In  some  cases,  the  periods  of  data  collection  differ 
for  the  on-site  and  for  the  Mitre-analyzed  statistics.  For  most  of  the 
statistical  summaries,  the  time  covered  by  this  report  (16  October  1978  -  7 
April  1979)  has  been  divided  into  26  periods.  Except  for  periods  1  and  14, 
these  are  all  7-day  periods.  In  addition,  the  users  have  been  assigned  to 
groups  according  to  functional  position  and  amount  of  use;  the  groups  are 
defined  in  section  3. 

Section  3  describes  the  use  of  the  MME  system  by  the  staff  officers  during 
their  normal  duty  periods.  Section  4  describes  the  use  of  the  MME  system  in  a 
Command  Post  Exercise.  Section  5  describes  the  problems  encountered  in 
interfacing  the  MME  system  to  AUTODIN,  and  Section  6  presents  the  conclusions 
reached  so  far  in  the  experiment.  Appendix  A  presents  the  observations  of  the 
CINCPAC  users,  and  Appendix  B  contains  data  tables  describing  the  use  of  the 
system. 


SECTION  3 


PATTERNS  OF  USE 


Tha  total  number  of  users  on  the  system  and  the  number  of  new  users 
introduced  each  week  from  16  October  1978  to  7  April  1979  are  presented  in 
Appendix  B,  Table  1.  The  total  number  of  users  increased  from  26  on  25  October 
to  89  on  17  March  and  then  appeared  to  reach  a  steady  state  of  70  on  7  April. 
The  maximum  number  of  new  users  per  week  was  10  in  early  January  followed  by 
8  in  early  November  and,  again,  in  early  March.  The  peak  use  occurred  during 
Exercise  Power  Play  79.  In  Figure  1,  two  metrics  are  plotted  to  indicate  the 
general  pattern  of  system  use  -  the  solid  line  indicates  the  number  of  users 
per  day  on  the  system  and  the  broken  line  indicates  the  average  daily  peak 
instructions  per  hour  for  each  of  the  periods.  There  was  a  period  of  system 
instability  in  the  early  part  of  January  that  is  reflected  in  the  lower  amount 
of  use.  The  number  of  instructions  per  hour  statistic  was  not  collected 
between  10  January  and  21  February.  The  system  use  increases  with  the  begin¬ 
ning  of  the  full  experimental  use  phase  and  peaks  during  the  exercise  Power 
Play  79.  The  data  indicate  that  the  system  use  is  stabilizing  after  the  exer¬ 
cise,  but  the  period  of  observation  isn't  long  enough  to  draw  that  conclusion. 

A  comparison  of  the  session  transcripts  for  the  beginning,  middle,  and  end 
of  the  reporting  period  reveals  that  there  is  a  consistent  core  of  experienced 
users.  In  the  week  of  19  October,  26  users  logged  on  the  system;  in  the  next 
week  (26  October),  23  of  these  same  users  were  seen  on  the  system.  Seven  new 
users  legged  on.  The  pattern  of  a  few  new  users  being  added  each  week  to  a 
consistent  core  of  users  continued  throughout  the  period. 

During  the  period  of  November  1978  to  April  1979,  the  average  number  of 
MME  sessions  per  week  per  user  did  not  change  significantly  (it  was  around  six 
sessions  per  week).  The  average  length  of  a  session  did  increase  (from  around 
1.5  hours  to  2  hours)  as  did  the  number  of  instructions  per  job.  Hence  these 
metrics  also  indicate  increasing  system  use  during  the  period. 

In  the  remainder  of  this  section,  the  data  that  were  collected  during  the 
limited  and  full  experimental  use  phases  are  discussed.  The  limited  experi¬ 
mental  use  (LEU)  period  examined  is  2  November  1978  through  10  January  1979 
(a  ten-week  period);  full  experimental  use  (FEU)  of  the  MME  system  began 
22  February  1979,  and  data  for  this  report  were  collected  through  7  April  1979 
(a  little  more  than  a  six-week  period). 

Patterns  of  use  among  different  types  of  users  are  of  interest  because  they 
illustrate  the  relationship  between  the  type  of  job  a  person  has  and  the  amount 
and  type  of  resources  needed  to  support  that  job.  To  facilitate  the  analysis 
of  the  relationship  between  system  use  and  job  position,  Sigma  users  have  been 
divided  into  groups  according  to  their  functional  positions.  The  groups  are: 

Division/Branch  Administrative  (DBADM)  -  clerical  personnel  with 
administrative  responsibility. 

J301  -  the  administrative  section  of  J3. 
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Action  Officers  (DBACT)  -  action  officers  are  further  divided  into 
categories  depending  on  the  volume  of  messages  handled. 

Division/Branch  Management  (DBMGT)  -  management  personnel  in  the 
division  and  branch  offices. 

Division/Branch  Clerical  (DBCL)  -  clerical  personnel  with  clerical 
responsibility  (as  compared  with  DBADM) 

Duty  Director  of  Operations  (DDO)  -  head  of  the  command  center  watch 
team. 

Air  and  Surface  Desk  Officers  (AIR/SUR)  -  on  command  center  watch 
team. 

Joint  Reconnaissance  Center  (JRC)  -  officers  in  the  JRC. 

3.1  AMOUNT  OF  USE 

Understanding  the  type  of  load  put  onto  a  system  by  different  types  of 
users  is  important  for  designers  of  future  systems.  For  some  types,  such  as 
J301  and  the  Command  Center  Watch  Team,  a  fairly  small  number  of  people  per 
shift  have  a  specific  set  of  message-related  tasks  to  perform.  Examination  of 
session  transcripts  and  patterns  of  use  data  show  that  these  types  of  users 
developed  consistent  daily  habits  when  using  Sigma.  To  determine  the 
resources  needed  to  support  these  users,  the  group  activity  should  be  analyzed. 

For  other  types  of  users,  the  load  put  on  the  system  by  the  individual  is 
more  important.  In  the  division  and  branch  offices,  and  among  the  action 
officers,  the  tasks  to  be  performed  have  more  day-to-day  variation.  System 
use  is  related  to  the  individual  office  message  load  and  the  events  of  the 
day.  To  support  these  offices  it  is  necessary  to  look  at  the  office,  the 
message  load,  and  the  number  of  people  in  the  office. 

The  different  approaches  needed  to  determine  necessary  support  for 
different  user  types  have  led  to  the  presentation  of  the  data  in  two  forms. 

One  shows  the  average  amount  of  Sigma  use  by  the  entire  group  of  users  in  a 

given  type;  the  other  shows  the  average  amount  of  use  by  an  individual  user  of 

a  given  type.  Table  10  of  Appendix  B  shows  in  detail  the  amount  of  on-line 
time  and  the  number  of  instructions  executed  for  the  different  types  of  users 
by  group  and  by  individual  during  each  LEU  and  FEU  period. 

3.1.1  ON-LINE  TIME 

The  amount  of  time  a  user  spends  logged  on  has  been  found  to  be  related  to 
terminal  placement  as  well  as  to  job  function.  In  areas  such  as  the  Command 
Center  and  division  and  branch  offices,  terminals  are  designated  for  primary 
use  by  a  particular  position.  These  users  are  generally  logged  on  throughout 
the  bulk  of  their  work  shifts,  even  though  they  may  not  use  the  system 
continuously.  In  areas  where  a  terminal  is  shared  by  many  users,  each  user's 

on-line  time  is  less;  the  users  log  on,  use  the  system,  but  then  often  have  to 

turn  the  terminal  over  to  another  user.  (An  exception  to  this  pattern  is 
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found  in  J301,  where  the  users  have  a  terminal  for  their  primary  use,  but  tend 
to  log  on,  distribute  the  messages  and  log  off  when  the  task  is  complete.)  In 
many  cases,  long  log-on  times  and  low  usage  indicate  that  the  user  is  keeping 
the  terminal  on  in  case  of  message  alerts,  as  in  the  Command  Center,  or  so 
that  he  can  respond  quickly  to  a  request  for  information,  as  in  the  division 
offices . 

In  determining  distribution  of  terminals  for  the  future,  several  factors 
must  be  considered.  These  are  the  importance  to  the  user  of  having  a  terminal 
readily  available  when  asked  for  information,  the  importance  of  his  receiving 
an  alert  as  soon  as  it  arrives,  and  the  importance  of  his  accomplishing  a  task 
in  a  given  amount  of  time  or  at  a  particular  time  of  day.  The  greater  the 
user's  dependence  on  terminal  availability  for  accomplishing  an  essential 
task,  the  more  important  it  is  for  the  user  to  have  a  terminal  dedicated  to 
his  use. 

3.1.2  SYSTEM  USE 

The  transition  from  LEU  to  FEU  did  result  in  generally  longer  on-line 
times;  however,  it  is  necessary  to  look  at  the  number  of  instructions  executed 
to  see  if  the  system  was,  in  fact,  receiving  more  use. 

Figure  2  shows  a  comparison  of  the  mean  number  of  instructions  executed 
per  hour  at  the  end  of  LEU  and  at  the  end  of  the  FEU  period  covered  in  this 
report.  With  the  exception  of  the  noon  hour,  the  mean  number  of  instructions 
increased  during  the  regular  work  day.  (During  the  off-hours  the  system  is 
used  primarily  by  the  Command  Center  and  JRC  personnel.  Individual  differences 
among  this  small  group  of  users  (four  per  shift)  is  the  most  likely  explanation 
of  the  variation  shown  during  off-hours  in  both  periods.)  The  increased  use  is 
due  partly  to  a  greater  number  of  users  on  the  system  and  partly  to  a  greater 
amount  of  use  by  the  core  group  already  familiar  with  Sigma. 

Directives  for  system  use  issued  at  the  beginning  of  FEU  had  the  greatest 
effect  on  J301,  the  Command  Center  duty  officers,  and  administrative  personnel 
in  the  division  offices.  J301  was  directed  to  distribute  early  in  the  day  the 
messages  that  had  accumulated  overnight;  they  were  also  directed  to  continue 
message  distribution  throughout  the  day.  The  greatest  increases  in  MME  system 
use  were  in  the  Air  and  Surface  desk  duty  officers'  activities.  The  Air  Desk 
duty  officers  were  directed  to  create  readboards  for  the  J3.  (These  readboards 
were  also  available  for  use  by  other  offices  within  J3.)  The  Surface  desk 
officers  responded  to  the  FEU  directives  by  processing  messages  on-line  when¬ 
ever  possible.  The  use  of  the  MME  readboards  by  division  chiefs  was  evident 
in  an  increased  level  of  activity  by  their  administrative  personnel,  who  use 
both  their  own  accounts  and  those  of  the  management  personnel  (DBMGT).  The 
support  personnel  responded  to  requests  from  their  chiefs  to  find  information, 
including  readboard  items,  for  them. 
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3.2  TYPE  OF  USE 


One  of  Che  goals  of  Che  MME  is  Co  decermine  what  capabilities  are 
necessary  in  an  automated  message  handling  sysCem  (AMHS)  and  to  whom.  IC  is 
imporcant  once  again  to  look  at  differences  in  the  styles  of  use  of  different 
types  of  CINCPAC  users.  Their  job  responsibilities  influence  the  instructions 
they  execute  on  Sigma  and  the  sequence  of  actions  they  perform. 

3.2.1  FUNCTION  UTILITY 

The  value  of  the  many  message-handling  functions  Sigma  offers  varies 
considerably.  Some  functions  are  used  by  nearly  everyone.  Others  are  used 
heavily  by  one  type  of  user  but  are  ignored  by  the  rest.  Other  instructions 
are  seldom  used,  but  are  critical  to  an  important  task.  Other  instructions 
appear  to  have  no  value  (at  best,  marginal  value)  for  any  users.  A  goal  of 
the  data  analysis  is  to  identify  into  which  of  these  categories  each  of  the 
Sigma  functions  falls. 

The  LEU  and  FEU  data  show  that  many  Sigma  instructions  are  executed  by  all 
types  of  users,  rather  than  being  used  particularly  by  any  one  group.  For 
example,  "restrict"  and  "augment",  which  enable  a  user  to  select  a  subset  of 
messages,  received  a  moderate  amount  of  use  by  all  types  of  users  during  LEU 
and  FEU.  These  instructions  and  similar  functions  can  be  considered  to  be 
generally  useful  for  future  systems. 

Relatively  few  of  the  instructions  are  used  heavily  by  only  one  type  of 
user.  Route  is  an  example  of  such  an  instruction.  This  instruction  performs 
four  functions  on  a  message  or  group  of  messages.  It  assigns  action  on  them, 
forwards  them  for  information,  files  them  in  designated  files,  and  deletes 
them  from  the  pending  file  if  designated  to  do  so.  Route  is  used  almost 
exclusively  by  J301  (the  section  responsible  for  message  distribution  within 
J3).  Route  represents  25-30Z  of  all  instructions  executed  by  J301.  During 
the  FEU  period  reported  here  (about  six  and  a  half  weeks),  J301  users  executed 
the  route  instruction  2603  times.  Without  route  they  would  have  executed  four 
to  ten  times  as  many  instructions  (and  would  have  taken  more  time)  to  perform 
the  same  tasks.  This  confirms  that  the  utility  of  a  function  must  be 
evaluated  in  terms  of  its  utility  to  each  group  of  users  rather  than  only  in 
terms  of  overall  use  by  the  entire  set  of  users. 

Finally,  there  are  instructions  that  are  seldom  used,  but  are  critical  to 
a  particular  task.  "Release  message"  is  such  an  instruction.  The  small 
number  of  outgoing  messages  sent  via  Sigma  resulted  in  light  use  of  release  by 
the  limited  set  of  personnel  given  release  authority.  Regardless  of  the 
amount  of  use  it  received,  however,  release  is  necessary  in  a  system  designed 
to  support  outgoing  message  processing. 

By  the  end  of  the  experiment  it  will  be  possible  to  categorize 
instructions  and  functions  by  their  relative  utility  to  the  different  types  of 
users  and  to  the  overall  group  of  users.  Some  instructions  may  turn  out  to  be 
of  marginal  use  to  everyone.  These  would  have  low  priority  in  the  design  of 
future  systems. 


3.2.2  STYLES  OF  USE 


Analyses  of  session  transcripts  and  interviews  with  the  users  have  shown 
that  there  is  a  relationship  between  a  user's  style  of  interaction  with  the 
system  and  his  position  in  the  organization. 

In  J301,  a  typical  session  begins  with  the  user  displaying  his  pending 
file,  a  file  containing  citations  for  all  the  incoming  messages.  (Each 
citation  contains  information  about  a  message:  originator,  date  and  time  of 
transmission,  precedence,  classification,  and  subject.)  He  first  identifies 
(through  the  use  of  previously  stored  selectors)  so-called  "junk"  messages 
which  do  not  have  to  be  distributed,  and  deletes  them.  Using  standard 
selectors,  he  then  identifies  various  sets  of  messages  with  common  routing 
criteria  (a  common  subject,  for  example).  Each  of  these  message  sets  is 
distributed  using  an  appropriate  routing  object.  After  all  the  messages 
satisfying  the  standard  selectors  have  been  distributed,  the  user  checks  the 
remaining  messages  and  distributes  them. 

Most  of  the  messages  are  distributed  on  the  basis  of  the  information 
contained  in  the  citations;  during  FEU,  only  about  7Z  of  the  messages 
distributed  were  viewed  before  the  distribution  decision  was  made.  J301 
chooses  to  view  messages,  rather  than  displaying  them,  because  viewing  does 
not  disrupt  the  list  of  message  citations  in  the  display  window.  This  type  of 
use  clearly  indicates  the  need  for  split-  (or  dual-)  screen  display. 

The  Air  desk  officers  have  a  similar  style  of  system  use,  based  on  their 
responsibility  for  creating  readboard  files.  First,  sets  of  selectors  are 
used  to  find  various  types  of  messages  appropriate  for  each  of  the  readboard 
files.  (This  is  analogous  to  using  standard  selectors  to  find  messages  with 
common  routing  criteria.)  After  each  set  is  selected,  the  user  files  those 
messages  in  the  appropriate  readboard  file  and  deletes  them  from  his  pending 
file.  When  all  the  pre-defined  sets  have  been  found,  the  user  scrolls  through 
the  remaining  message  citations  one  at  a  time,  deleting  each  citation  as  it  is 
scanned.  The  officers  choose  to  delete  these  messages  individually,  rather 
than  as  a  group,  to  ensure  that  all  have  been  seen.  As  a  result,  the  Air  desk 
officers  execute  a  high  proportion  of  delete  message  instructions. 

Other  types  of  users  have  less  structured  tasks  to  perform.  While  an 
individual  member  of  one  of  these  types,  action  officers  or  clerks  for 
example,  may  develop  consistent  habits  from  day  to  day,  as  a  group,  these 
types  have  more  variation  in  their  system  use  than  J301  and  the  Air  desk  duty 
officers.  One  action  officer  may  use  Sigma  only  for  reading,  printing,  and 
deleting  messages,  while  another  might  take  advantage  of  selectors  and  other 
file  and  message  manipulation  functions. 

There  are  several  points  of  interest  regarding  all  types  of  users.  The 
choice  of  "display"  or  "view"  to  look  at  a  message  is  primarily  a  matter  of 
personal  preference;  the  two  instructions  are  used  almost  equally.  Some  users 
choose  to  view  messages,  which  enables  them  to  keep  a  list  of  the  message 
citations  in  the  "display  window"  (the  upper  portion  of  the  CRT)  at  the  same 
time.  This  list  can  be  manipulated  while  a  message  is  being  viewed.  Other 
users  choose  to  display  messages  and  rarely  view  them.  A  need  to  edit  the 
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message  and  the  brighter  intensity  of  the  display  window  are  factors 
influencing  this  choice.  It  is  clear  from  the  amount  of  use  that  both  display 
and  view  receive  that  both  are  useful  capabilities.  A  split  screen  option, 
and  the  ability  to  look  at  a  message,  text  object,  or  file  list  in  one  window 
without  losing  the  ability  to  manipulate  an  object  in  the  other  window,  are 
capabilities  that  should  be  included  in  future  system  designs. 

It  is  also  apparent  from  the  relatively  low  number  of  messages  displayed 
and  viewed,  in  comparison  to  the  number  delivered  to  their  pending  files,  that 
the  users  are  making  many  decisions  about  the  disposition  of  their  messages  on 
the  basis  of  the  message  citation.  .Because  paper  copies  of  the  messages  are 
being  distributed  in  parallel  with  Sigma  distribution,  this  point  will  be 
probed  further  during  the  final  interviews.  At  present  it  is  not  clear  if  the 
users  recognize  the  messages  from  having  previously  seen  the  paper  versions, 
or  if  the  citation  does  contain  sufficient  information  for  their  decision 
making.  (In  J301  recognition  is  not  a  factor;  these  users  have  already  stated 
that  the  citation  contains  sufficient  information  to  permit  direct 
distribution  of  932  of  the  messages.) 

3.2.3  FLAGGED  INSTRUCTIONS 

Instruction  flags  are  artifacts  of  the  data  collection  facility.  They  are 
recorded  in  the  data  collection  files  when  a  user  tries  to  execute  an 
instruction  that  cannot  be  completed  successfully.  They  may  indicate  an  error 
on  the  part  of  the  user,  or  they  may  indicate  the  system's  inability  to 
complete  (from  a  system  viewpoint)  a  legitimate  instruction. 

The  greatest  number  of  flags  occur  when  the  user,  processing  a  set  of 
messages  after  having  already  processed  the  last  message,  tries  to  execute  an 

instruction  to  process  the  next  message  in  the  set.  When  this  happens  the 

system  returns  a  "you  are  currently  at  the  end  of  the  file”  message.  While 
this  does  not  distress  the  users,  who  have  become  accustomed  to  interpreting 
it  as  an  "end  of  list  message",  it  does  cause  some  delay.  A  better  design 
would  be  to  provide  an  end  of  list  indicator  when  the  last  item  of  a  list  is 
being  processed. 

Another  common  flag  occurs  when  the  user  tries  to  process  an  open  object 
and  none  is  open.  Evidently  users  lose  track  of  the  status  of  various  types 

of  objects  even  though  an  indication  is  provided  in  the  status  line  at  the  top 

of  the  screen.  This  point  will  be  investigated  further  in  the  final 
interviews . 

Two  types  of  flags  occur  frequently  because  of  system  status.  One  is  the 
failure  to  display  a  file  or  message  because  that  object  is  being  updated  or 
because  a  message  has  been  archived.  The  other  is  the  failure  of  a  restrict 
or  augment  instruction  to  result  in  entries  that  satisfy  the  specified 
parameters.  In  the  former  case,  the  unavailability  of  an  object  due  to  update 
status  or  previous  archival  forces  the  user  to  decide  if  seeing  the  object  is 
worth  the  delay.  In  the  latter  case,  the  appearance  of  the  message  "your 
selector  returned  no  entries"  is  regarded  by  the  users  as  a  satisfactory 
system  response;  it  requires  no  further  action  by  the  user.  In  both  cases,  as 
far  as  the  user  i3  concerned,  the  system  has  responded  to  his  instruction;  no 
errors  have  been  made. 
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SECTION  4 


EXERCISE  PARTICIPATION 


The  MME  system  was  used  during  the  JCS  Exercise  Power  Play  79.  The  main 
objectives  of  the  MME  participation  were  to  develop  procedures  for  the  use  of 
an  automated  message  handling  system  in  exercise/crisis  situations  and  to 
determine  the  usefulness  of  such  a  system  in  exercises  and  crises. 

4.1  BACKGROUND 

This  exercise  was  a  24-hour  per  day  Command  Post  Exercise  (CPX)  conducted 
from  6  to  23  March  1979.  A  CPX  is  an  exercise  involving  the  Commander,  his 
staff,  and  communications  within  and  between  headquarters.  CINCPAC  participa¬ 
tion  in  the  exercise  was  minimal;  there  was  a  skeleton  Operations  Action  Group 
(OAG),  but  there  was  no  Operations  Planning  Group  and  no  Logistics  Readiness 
Center. 

The  Staff's  basic  approach  to  a  crisis  is  to  form  crisis  action  teams 
comprised  of  a  Crisis  Action  Coordinator,  a  Deputy  Crisis  Action  Coordinator, 
a  Current  Operations  Support  Element,  an  Operations  Action  Group,  an  Operations 
Planning  Group,  and  a  Logistics  Readiness  Center.  The  Operations  Action  Group 
is  composed  of  the  Deputy  Crisis  Action  Coordinator,  the  Operations  Action 
Group  Executive  Officer,  and  representives  from  staff  directorates.  It 
functions  as  a  coordination  point  for  unifying  the  efforts  of  the  Operations 
Planning  Group,  Command  Center  Watch  Team,  Current  Operations  Support  Element, 
Logistics  Readiness  Center,  and  significant  support  actions.  The  Operations 
Action  Group  is  located  in  the  CINCPAC  Command  Center.  For  additional 
description  of  the  OAG  see  pages  35  and  36  in  appendix  A. 

4.2  PRE-EXERCISE  SET-UP  AND  TRAINING 

Fifty  MME  accounts  were  identified  and  established.  These  included  the 
Crisis  Action  Coordinator,  Deputy  Crisis  Action  Coordinator,  Chief  of  the 
Operations  Action  Group,  Executive  Officer,  Assistant  Executive  Officer,  Jl 
(personnel),  J2  (Intelligence),  J3  (Operations),  J4  (Logistics),  J5  (Plans), 

J6  (Communications  and  Data  Processing),  J74  (Public  Affairs),  Exercise 
Controller,  Operations  Action  Group,  Logistics  Readiness  Center,  and  Current 
Operations  Support  Element.  The  exercise  s.ccounts  were  opened  as  one  group 
(i.e.,  each  account  was  able  to  access  all  files,  text  objects,  and  selectors 
in  the  other  accounts).  All  accounts  were  opened  with  release  authority  for 
informal  notes.  The  Deputy  Crisis  Action  Coordinator  and  the  Chief  of  the 
Operations  Action  Group  had  release  authority  for  formal  memoranda  and  AUTODIN 
messages. 

Personnel  from  th*»  CINCPAC  directorates  Jl,  J2,  J3,  J4,  J5,  J6,  and  J74, 
and  from  IPAC  (Intelligence  Center  Pacific)  who  were  potential  Operations 
Action  Group  members  were  identified  together  with  their  message-handling 
tasks.  These  personnel  were  trained  to  use  the  MME  system  with  emphasis  on 
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the  tasks  they  were  expected  to  perform  during  the  exercise.  The  training 
began  1  February  1979  and  continued  throughout  the  month  of  February.  It 
consisted  of  a  one-hour  introductory  lecture  to  groups  of  users,  with  three 
hours  a  day  available  for  hands-on  training  and  experience.  Both  officers  and 
administrative  personnel  were  trained. 

4.3  MME  PARTICIPATION 

Prior  to  and  during  the  first  week  of  the  exercise,  the  procedures  for  the 
use  of  the  MME  system  during  the  exercise  were  determined  and  implemented. 

The  effort  included  determining  and  establishing  the  files,  text  objects,  and 
selectors  required  by  the  MME  Executive  Officer  for  effective  use  of  the  MME 
system  during  the  exercise.  The  remaining  members  of  the  exercise  team  were 
free  to  establish  their  own  objects  on  an  individual  basis.  0 

The  MME  system  was  used  as  a  secondary  message-processing  system  on  a 
non-interference  basis.  The  MME  participation  commenced  at  the  beginning  of 
the  exercise  and  continued  throughout  the  exercise. 

The  MME  Executive  Officer  duplicated  the  functions  of  the  regular 
Executive  Officer,  who  utilized  the  paper  message-handling  system.  The  MME 
system  was  used  for  message  distribution,  retrieval,  coordination,  and 
release.  The  initial  plan  called  for  directorate  Action  Officers,  as  time 
permitted,  to  duplicate  on  the  MME  their  paper-system  actions.  When  it  was 
established  that  the  MME  system  was  in  a  stable  operating  condition,  they  were 
encouraged  to  and  did  use  the  MME  system  as  their  primary  system  for  certain 
outgoing  messages,  e.g.,  those  with  a  24-hour  or  longer  suspense  time. 

To  support  their  exercise  functions,  the  MME  Executive  Officers  created 
and  maintained  15  files,  36  text  objects,  and  2  selectors.  The  15  files 
included  a  master  file  for  all  messages  received  by  the  Operations  Action 
Group,  a  Status-of-Action  Log  (SAL),  a  Significant  Events  Log  (SEL),  and 
various  files  for  situation  reports,  JCS  Action  messages,  Operations  Action 
Group  action  requests  to  the  JCS  or  other  Headquarters,  CINCPAC  directives  to 
subordinate  headquarters,  MME  transmitted  messages,  paper-system  transmitted 
messages,  readdressals ,  backcopies,  etc. 

The  36  text  objects  were  used  mostly  to  ROUTE  incoming  messages  to 
different  combinations  of  the  exercise  accounts  for  Action  and/or  Info,  FILE 
in  the  Master  file,  and  DELETE  from  the  Executive  Officer  Pending  file. 

The  two  selectors  were  the  ALERT_SELECTOR  that  selected  and  alerted  the 
MME  Executive  Officer  on  all  incoming  messages  that  were  FLASHOVERRIDE  or 
FLASH  or  IMMEDIATE  or  FOR_RELEASE  or  FOR_CHOP  or  ACTION,  and  a  selector  used 
to  screen  files  for  NOTEs. 

The  following  paragraph  describes  the  message  handling  procudures  used  by 
the  MME  Executive  Officer  (MXO) ,  the  Deputy  Crisis  Action  Coordinator  (DCAC), 
and  the  Action  Officer  (AO). 
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All  incoming  messages  on  the  MME  came  to  the  OAG  Pending  File  (OAG  MXO). 

The  MXO  reviewed  each  incoming  message  citation  (or  message,  if  necessary). 

If  there  were  an  OAG  action  required,  he  assigned  action  on  the  message  to  the 
appropriate  OAG  action  officer  (AO)  which  automatically  updated  the  SAL.  If 
the  message  were  action  on  a  COG  item,  he  forwarded  it  to  the  appropriate  OAG 
AO.  All  incoming  messages  were  filed  in  the  OAG  Master  file.  Important 
messages  were  forwarded  to  the  DCAC.  The  MXO  maintained  and  updated  the  SEL 
as  required.  The  MXO  reviewed  all  outgoing  messages,  ensured  all  chops  had 
been  made,  resolved  any  non-concurrences,  and  forwarded  the  messages  to  the 
DCAC  for  release.  If  the  outgoing  message  satisfied  a  pending  OAG  action,  the 
MXO  updated  the  SAL.  If  the  outgoing  message  directed  a  subordinate  to  act  or 
requested  action  from  a  co-lateral  or  higher  headquarters,  the  MXO  posted  the 
appropriate  files.  All  outgoing  OAG  messages  were  filed  from  the  OAG  Pending 
File  to  one  of  the  three  outgoing  files.  The  MXO  ensured  that  all  actions 
generated  from  non-message  sources  were  entered  into  the  SAL  and  SEL  as 
appropriate.  The  MXO  readdressed  messages  as  directed  by  the  DCAC.  A 
terminal  was  available  for  the  DCAC,  but  was  rarely  used.  Releasing  outgoing 
messages,  especially  readdressals ,  was  the  DCAC's  primary  use  of  the  system. 

The  AOs  had  two  terminals  available  for  their  exclusive  use,  in  addition  to 
being  able  to  use  the  DCAC's  terminal  most  of  the  time.  Many  AOs  made 
extensive  use  of  the  MME  system  during  the  time  available  to  them  when  they 
were  not  processing  actions  using  the  manual  system.  The  J2  and  J4  AOs 
created  extensive  files  to  support  the  wide  variety  of  messages  received 
during  the  exercise. 

A  summary  of  activity,  system  availability,  and  traffic  handled  during  the 
exercise  is  given  in  Appendix  B,  Table  8. 

Normal  administrative  use  of  the  system  continued  throughout  the  exercise. 
In  order  to  make  the  system  more  responsive  to  the  exercise  users,  the 
scheduling  algorithm  was  changed  to  give  the  exercise  users  higher  priority 
than  the  other  users.  Even  so,  there  was  a  noticeable  degradation  in  system 
response  time  during  the  period  0600-0900  on  normal  work  days. 

A  CINCPAC  SITREP  was  required,  on  a  24-hour  basis,  throughout  the  exercise. 
The  one-time  development  of  the  SITREP  on  the  system  demonstrated  the  utility 
of  interactive  capabilities  within  the  system.  The  different  directorates 
prepared  their  portions  according  to  a  fixed  form  and  forwarded  them  to  the 
MME  Executive  Officer  who  assembled  them,  prepared  the  message,  and  forwarded 
the  message  to  the  DCAC  for  release. 

4.3.1  PERSONNEL 

The  personnel  actually  assigned  to  the  exercise  team  were  not  those 
previously  identified  and  trained.  Thus,  the  personnel  reporting  at  the 
commencement  of  the  exercise  had  to  be  trained  by  the  MME  Executive  Officers 
and  the  MME  Observer.  Those  personnel  reporting  later  were  trained  mostly  by 
the  initial  members  of  the  exercise  team  with  assistance  from  the  MME  Executive 
Officers  and  Observer.  In  spite  of  the  lack  of  previous  exposure  to  the  MME 
system,  they  quickly  learned  to  use  the  MME  in  carrying  out  many  of  their 
specific  duties.  For  this  exercise,  the  Operations  Action  Group  was  activated 
with  the  Deputy  Crisis  Action  Coordinator,  Executive  Officer,  MME  Executive 
Officer,  and  Action  Officers  from  the  J2,  J3,  J4,  J5  and  J6  directorates. 
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4.3.2  HIGHLIGHTS 


Even  chough  there  was  an  excessive  amount  of  system  downtime,  the  MME 
system,  when  available,  was  able  to  process  and  distribute  messages  in  a 
timely  manner.  The  readdressal  feature  was  enthusiastically  received  by  the 
Operations  Action  Group  Action  Officers  and,  once  demonstrated,  was  used  for 
almost  all  message  readdressals.  During  periods  of  low  Operations  Action 
Group  activity,  some  outgoing  messages  were  created  and  released  on  the  MME 
system.  On  one  occasion,  the  C1NCPAC  SITREP  was  prepared  on  various  Action 
Officer  terminals,  assembled  by  the  MME  Executive  Officer  on  his  terminal  and 
released  by  the  Deputy  Crisis  Action  Coordinator  on  his  terminal.  The  ability 
to  retrieve  messages  from  files  was  the  system  feature  for  which  the  users 
expressed  the  highest  degree  of  preference. 

4.3.3  PROBLEMS 

The  major  problem  with  the  use  of  the  MME  system  was  the  unacceptable 
amount  of  unscheduled  system  downtime.  The  lack  of  system  reliability 
mandates  a  backup  or  redundant  message-handling  system.  Initially  a  very  high 
load  average  caused  by  a  software  failure  resulted  in  lengthy  delays  in 
executing  commands.  Delays  also  occurred  during  the  coordination  or  release 
process.  On  one  occasion,  it  took  over  an  hour  for  a  message  to  get  from  one 
OAG  terminal  through  the  processing  cycle  and  arrive  at  another  Operations 
Action  Group  terminal. 

The  non-J3  members  of  the  Operations  Action  Group  were  inexperienced  in 
the  use  of  the  MME  system  and  the  terminal.  Although  a  training  program  had 
been  established  in  January  1979  specifically  for  non-J3  Operations  Action 
Group  members,  most  of  the  non-J3  members  who  actually  participated  in  the 
exercise  were  not  fully  qualified  on  the  system.  This  was  overcome  by 
intensive  on  the  job  training.  Within  a  few  sessions  the  non-J3  Operations 
Action  Group  members  were  capable  of  using  the  system  to  display,  file, 
retrieve,  create,  and  comment  on  messsges  with  minimal  assistance. 

The  release  of  OAG-originated  and  readdressed  messages  was  hampered  by 
previously  unknown  interface  problems  between  Sigma  and  the  LDMX.  A  related 
problem  was  the  lack  of  notification  to  the  user  when  a  message  was  rejected 
by  the  LDMX. 

The  MXO,  who  was  fully  trained  and  experienced  on  the  MME  system,  was 
totally  occupied  in  trying  to  accomplish  all  the  tasks  he  was  supposed  to 
perform  during  the  periods  he  was  working  on  the  MME  system.  Some  of  these 
tasks  were  not  ones  that  would  need  to  be  accomplished  in  the  normal  prosecu¬ 
tion  of  an  exercise,  but  were  tasks  designed  to  aid  in  the  evaluation  of  the 
MME  during  this  particular  exercise.  At  times  he  was  not  able  to  keep  up  with 
every  task  and  had  to  postpone  some,  such  as  maintaining  the  SAL  or  other 
files  containing  suspense  items.  The  MXO  was  able  to  catch  up  with  incoming 
message  distribution  in  approximately  one  hour  after  being  absent  for  an 
extended  period  (up  to  8—10  hours),  due  to  the  low  message  arrival  rate.  Since 
there  were  relatively  few  outgoing  messages  processed  on  the  MME  system,  the 
MXO  did  not  spend  much  time  on  this  facet  of  message  handling.  The  MXO  did 
spend  some  time  assisting  the  other  OAG  users.  This  would  not  be  required  if 
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all  OAG  members  were  fully  trained.  The  MXOs  agreed  that  one  MXO  per  12-hour 
shift  would  not  have  been  able  to  handle  all  the  tasks  assigned  to  the  MXO  for 
Power  Play  79  if  the  incoming  message  flow  had  been  comparable  to  that  of 
Exercise  Nifty  Nugget  -  given  the  problems  that  existed  with  the  MME  system 
during  the  exercise.*  But,  providing  that  the  system's  reliability  and 
responsiveness  can  be  improved,  the  MME  system  should  be  of  great  assistance 
to  the  MXOs  in  several  of  their  message-handling  tasks.  The  MME  system's 
speed  and  filing  system  permit  much  more  efficient  maintenance  of  the  master 
message  file  and  log,  the  SAL,  and  the  SEL. 

The  highlights  and  problems  associated  with  the  use  of  the  MME  system  in 
PP-79  are  covered  in  greater  detail  in  Appendix  A. 

4.4  MME  SYSTEM  EFFECTIVENESS 

A  user's  perception  of  the  effectiveness  of  a  system  is  based  on  whether 
or  not  it  helps  him  do  his  job  better  or  easier.  This  is  influenced  by  what 
he  sees  as  the  objective  of  the  exercise,  by  the  type  of  training  he  has 
received,  by  the  system's  reliability  and  availability,  and  by  the  system's 
capability  and  efficiency. 

A  batch-processing  system  may  be  down  a  few  hours  without  a  user  realizing 
it;  he  may  simply  get  his  messages  a  bit  later.  Users  are  acutely  aware  of 
all  the  reliability/availability  problems  of  an  interactive  system  that  they 
are  using  and  depending  on.  The  reliability  requirements  for  an  automated 
message  handling  system  in  a  crisis  or  exercise  are  very  high. 

There  were  three  factors  that  prevented  this  exercise  from  serving  as  a 
satisfactory  test  for  determining  the  usefulness  of  an  automated  message 
processing  system  in  an  exercise/crisis  situation.  First,  CINCPAC 
participation  in  the  exercise  was  minimal;  second,  the  MME  system  was  used  as 
the  secondary  message-processing  system  on  a  non-interference  basis;  third, 
there  were  periods  during  the  exercise  when  a  software  problem  caused 
excessive  system  delay. 

Timing  statistics  were  recorded  during  one  period  of  the  exercise  in  order 
to  compare  the  message  arrival  times  for  copies  of  messages  sent  directly  and 
copies  sent  via  the  MME  system. 

Upon  receipt  of  a  high-precedence  message  for  the  Operations  Action  Group, 
the  LDMX  routes  it  to  the  MME  system  and  directly  to  a  printer  in  the 
Operations  Action  Group  spaces.  Lower  precedence  messages  and  back-up  copies 
of  high  precedence  messages  are  sent  from  the  Communications  Center  via  a 
pneumatic  tube.  For  the  37  messages  that  were  received  during  the  period,  the 
average  arrival  time  via  the  MME  was  3.4  minutes  later  than  the  arrival  time 
on  the  line  printer.  The  arrival  time  of  the  back-up  copies  and  the  lower 
precedence  messages  via  the  tube  was  2  to  29  minutes  later  than  the  arrival 
via  the  MME. 


*Exercise  Nifty  Nugget  was  conducted  late  in  calendar  year  1978.  The  traffic 
load  was  substantially  higher  than  that  in  Power  Play  79. 
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INTERFACE  TO  AUTODIN 


The  designers  of  future  automated  message  handling  systems  must  be 
sensitive  to  the  problems  that  arise  because  of  the  lack  of  an  integrated 
design  for  the  total  conanunications  system.  In  the  military  message 
experiment,  some  of  the  features  that  the  users  see  are  provided  by  the  MME 
system  and  some,  e.g.,  plain  language  address  translations,  transmissions  to 
the  AUTODIN  network,  and  initial  distribution  to  the  J3  Directorate,  are 
provided  by  the  Local  Digital  Message  Exchange  (LDMX).  Although  there  are 
recognized  inefficiencies  in  the  MME  hardware  and  software,  there  are  strong 
indications  that  the  type  of  service  provided  to  users  by  the  MME/LDMX 
combination  will  require  more  computer  power  (in  processor  cycles  and  memory) 
than  was  previously  estimated.  And  while  it  is  realized  that  in  future 
systems  different  architectures  for  distributing  the  computing  power  will  be 
used,  there  will  be  a  need  to  perform  in  an  integrated  manner  (from  the  user's 
view)  those  functions  that  are  now  performed  by  the  MME  system  and  by  the 
LDMX.  The  observations  during  this  phase  of  the  MME  lead  to  the  conclusion 
that  there  must  be  an  integrated  design  for  the  total  communications  system. 

The  LDMX  was  developed  by  the  Navy  under  its  automation  program.  The 
procedures  and  software  in  the  LDMX  are  used  at  many  Navy  communications 
facilities,  and  any  changes  are  closely  controlled  by  COMNAVTELCOM  to  ensure 
the  integrity  of  the  system.  Because  of  this,  changes  to  the  LDMX  take  a  long 
time  to  be  approved  and  implemented  (as  they  should  in  an  operational  system), 
and,  thus,  most  of  the  changes  to  correct  problems  in  the  LDMX/MME  interface 
are  made  in  the  MME  system.  Because  some  of  the  problems  cannot  be  solved  in 
just  the  MME  system,  they  remain  outstanding.  Some  of  the  problems  that  must 
be  considered  by  the  designers  of  future  systems  are  discussed  in  Section  5.2. 

5.1  SUMMARY  OF  AUTODIN  TRAFFIC 

5.1.1  INCOMING  MESSAGES 

During  the  period  16  October  to  7  April,  the  MME  System  processed  a  total 
of  113,597  AUTODIN  messages.  As  can  be  seen  by  the  message  traffic,  (see 
Table  4,  Appendix  B)  the  weekly  average  increased  from  approximately  3,000 
messages  per  week  to  just  over  5,000  messages  per  week.  A  peak  of  8,000 
messages  was  reached  during  the  week  of  11-17  March  during  the  exercise.  The 
percentage  of  J3  Action/Cognizance  messages  ranged  from  30-50X.  The  main 
contributing  factor  to  both  the  increase  of  average  messages  received  and  the 
lowering  of  the  J3  Action/Cog  percentages  was  the  reintroduction  of  Foreign 
Broadcast  Information  Service  (FBIS)  type  messages  on  the  system.  This  was 
done  in  late  November. 

5.1.2  OUTGOING  MESSAGES 

General  use  of  the  MME  system  to  transmit  messages  was  authorized  to  begin 
on  22  February  1979.  At  that  time,  the  system  was  operating  with  general 
service  traffic  up  to  the  SECRET  level.  Some  constraints  were  placed  on  the 
initial  use  of  the  MME  system  to  transmit  messages.  The  MME  system  was  used 
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to  prepare  and  transmit  an  out-going  message  if  a)  the  message's  classification 
was  no  greater  than  SECRET,  b)  there  were  at  least  two  working  days  prior  to 
the  message's  suspense  date,  c)  all  items  referenced  by  the  message  were  on 
the  MME  system,  and  d)  all  coordinating  officers  were  MME  system  users.  From 
that  date  to  7  April  1979,  228  outgoing  AUTODIN  messages  were  created  and 
released  by  J3  users  on  the  MME  system.  In  general,  about  402  of  the  messages 
released  on  the  system  were  original  narrative  and  602  were  readdressals  of 
incoming  messages.  In  comparison,  it  appears  that  about  452  of  all  J3 
outgoing  message  traffic  are  readdressals.  Message  creation  and  release 
peaked  during  the  exercise.  Two  weeks  after  the  exercise  (25-31  March),  382 
of  J3's  outgoing  traffic  was  created  and  released  on  the  MME  system. 

The  messages  released  on  the  MME  System  during  the  period  of  observation 
tended  to  be  shorter  than  the  average  messages  released  by  the  J3  Directorate. 
Use  of  the  system  for  release  tends  to  follow  the  typical  day-of-the-week 
pattern  seen  in  the  world  of  paper  messages. 

5.2  MME/LDMX  INTERFACE 

The  use  of  the  MME  system  for  outgoing  traffic  starting  in  late  February 
1979  and  intensifying  during  the  exercise  participation  in  March  revealed  a 
number  of  incompatibilities  in  the  MME/CINCPAC  LDMX  interface.  These 
incompatibilities  caused  problems  that  vary  in  seriousness.  Some  could 
require  manual  intervention  by  LDMX  personnel  and  result  in  delayed  delivery; 
others  could  result  in  non-delivery  of  messages. 

5.2.1  INCOMING  MESSAGES 

This  section  discusses  the  handling  of  special  categories  of  messages  such 
as  TOP  SECRET  and  Limited  Distribution  messages  and  the  general  forced  delivery 
of  messages  to  the  MME  System.  Because  of  current  LDMX  operating  procedures 
and  software  design,  it  is  not  possible  to  deliver  TOP  SECRET  messages  to  the 
MME  system. 

Frequently,  a  member  of  the  CINCPAC  Staff  will  determine  that  J3  should 
receive  a  copy  of  an  incoming  message  on  which  J3  was  not  included  in  the 
initial  distribution.  In  such  a  case,  the  staff  member  submits  to  the  LDMX  a 
"request  for  a  change  in  internal  distribution"  in  which  he  adds  J3  and  any 
other  office  codes  that  he  feels  are  appropriate.  The  LDMX  recalls  the 
message  and  performs  a  "distribution  edit."  When  the  edit  function  is 
executed,  the  LDMX  can  provide  delivery  of  the  message  only  to  a  printer.  The 
LDMX  cannot  cause  delivery  of  the  message  to  the  MME  system.  This  results  in 
"missing  messages"  in  the  MME  data  base;  i.e.,  J3  users  receive  some  messages 
via  the  paper  system  that  are  not  received  from  the  MME  system.  A  method  to 
force  delivery  of  the  message  to  the  MME  system  in  this  case  is  necessary. 

5.2.2  OUTGOING  MESSAGES 

The  LDMX  requires  that  outgoing  AUTODIN  messages  adhere  to  a  prescribed 
format.  There  have  been  numerous  occasions  when  the  MME  system  sent  outgoing 
AUTODIN  messages  to  the  LDMX  in  improper  format;  some  resulted  in  LDMX 
crashes.  The  major  format  problems  have  been  corrected. 


5.2.3  RETRANSMITTALS 


When  the  LDMX  has  rejected  a  message  sent  to  it  by  the  MME  system,  it  is 
no  longer  editable  in  the  MME  system  nor  can  it  be  released  again.  Currently, 
it  is  not  possible  to  retransmit  a  released  message  from  the  MME  system  to  the 
LDMX  without  creating  an  entirely  new  message  and  repeating  the  release 
process.  The  MME  system  should  provide  the  capability  for  the  MME  operator 
and/or  the  users  to  correct  and  retransmit  released  messages  that  were 
rejected  by  the  LDMX. 

5.2.4  MULTISECTION  MESSAGES 

The  LDMX  does  not  pass  the  sections  of  incoming  multisection  messages  to 
the  MME  system  consecutively  in  order.  The  MME  system,  in  turn,  is  unable  to 
recognize  the  sections  of  multisection  messages;  consequently,  incoming  multi¬ 
section  messages  may  appear  in  the  MME  files  out-of-order  and  interspersed 
among  other  messages. 

In  the  case  of  outgoing  messages  (paper  and  MME  systems),  the  LDMX 
sectionalizes  long  messages,  assigning  the  same  DTG  to  all  sections  of  the 
message.  For  readdressals  in  the  paper  system,  a  separate  Form  DD173  is 
submitted  for  each  section  of  the  readdressed  message,  the  LDMX  recognizes  the 
various  sections  of  the  readdressal,  and  it  assigns  the  same  DTG  to  each 
section  of  the  outgoing  message.  However,  SIGMA  assigns  a  unique  DTG  to  each 
outgoing  message  it  forwards  to  the  LDMX.  SIGMA  has  no  capability  to 
recognize  multisection  readdressals  and,  therefore,  does  not  assign  the  same 
DTG  to  each  section  of  a  multisection  message.  The  LDMX,  in  turn,  recognizes 
an  inconsistency  in  this  situation  and  rejects  the  readdressal  with  a  Service 
Message  to  the  MME  system. 

5.2.5  READDRESSALS 

Whenever  the  size  of  the  LDMX  on-line  data  base  reaches  a  prescribed 
number  of  pages,  they  archive.  In  LDMX  jargon,  this  is  called  a  database 
wrap-around.  Currently,  wrap-around  involves  messages  that  have  been  on-line 
for  15  days  or  more.  When  an  MME  user  initiates  a  request  to  readdress  one  of 
these  archived  messages,  the  LDMX  rejects  it  with  a  Service  Message  to  the 
MME.  This  may  cause  a  non-delivery  of  the  message  being  readdressed. 

5.2.6  UNCLASS IF IED-EFTO  MESSAGES 

CINCPAC  users  frequently  need  to  originate  Unclassified  Encrypted  for 
Transmission  Only  (UNCLAS  EFTO)  messages.  Currently,  SIGMA  permits  the  user 
to  create  an  UNCLAS  message  and  to  then  edit  the  classification  line  (in  format 
line  twelve  of  the  message)  to  read  UNCLAS  EFTO.  However,  SIGMA  does  not 
change  the  classification  indicators  in  format  lines  two  and  four.  Because  of 
the  classification  mismatch  between  format  lines  two,  four,  and  twelve,  the 
LDMX  rejects  the  message  and  sends  a  Service  Message  citing  the  security 
mismatch.  This  deficiency  results  in  manual  intervention,  unnecessarily- 
delayed  delivery,  and  potential  non-delivery  of  the  message. 
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5.2.7  SERVICE  MESSAGES 


When  the  LDMX  receives  a  message  which  it  cannot  process,  it  returns  an 
abbreviated  message  called  a  Service  Message.  This  Service  Message  identifies 
the  message  in  question  and  states  the  reason  for  nonprocessing.  The 
appropriate  addressee  for  the  Service  Message  must  be  determined.  In  some 
cases,  it  is  appropriate  for  the  LDMX  to  handle  them;  in  other  cases  the  MME 
system  or  message  drafter  needs  to  be  consulted.  Currently,  the  MME  system 
contains  no  provisions  for  handling  Service  Messages.  This  results  in  a 
potential  non-delivery  of  outgoing  messages  released  by  the  MME  system. 

5.2.8  LACK  OF  UPWARD  COMPATIBILITY  IN  LDMX  RELEASES 

NAVCOMPARS  software  version  9.0  is  scheduled  for  installation  in  the 
CINCPAC  LDMX.  As  a  result  of  this  change,  the  LDMX  will  lose  the  capability 
to  recall  and  retransmit  messages  by  Channel  Sequence  Number  (CSN).  This 
feature  has  a  tremendous  amount  of  utility  in  the  LDMX/MME  interface  and  its 
use  is  referred  to  as  a  Channel  Rerun  Request  (CRR).  In  order  to  ensure  the 
integrity  of  the  MME  message  data  base,  the  MME  operator  routinely  scans  the 
Channel-Log  in  search  of  missing  CSNs.  The  missing  CSNs  are  then  passed  by 
telephone  to  the  LDMX  Command  VDT  operator  as  a  CRR  command,  and  the  messages 
are  retransmitted  to  the  MME  system.  Without  this  feature,  there  is  no 
assurance  that  a  message  from  the  LDMX  to  the  MME  has  not  been  lost. 

5.2.9  MESSAGE  FORMATS 

Currently,  there  is  no  jointly  agreed-upon  rigorous  format  control  for 
subject  line,  references,  key-words,  and  pass-to  instructions.  A  message  that 
is  passed  to  the  MME  system  from  the  LDMX  contains  the  distribution 
information  that  the  LDMX  has  generated  and  that  portion  of  the  received 
message  that  is  defined  in  ACP-126.  Thus,  the  text  field  contains,  in 
addition  to  the  text,  information  that  is  needed  for  intra-Directorate 
distribution.  In  order  that  future  message  processing  systems  be  able  to 
automate  many  of  the  functions  that  are  now  done  manually,  ACP-126  should  be 
modified  to  add  discipline  to  the  handling  of  pass-to  instructions, 
references,  keywords,  and  subject. 

5.2.10  DUPLICATIVE  UNCOORDINATED  CONTROL  CHECKS 

Most  of  the  problems  encountered  in  the  LDMX/MME  system  interface  are  the 
results  of  data  stored  redundantly  and  checks  made  independently  in  the  two 
systems.  This  results  in  a  system  that  presents  sometimes  confusing  and 
contradictory  responses  to  the  user.  These  problems  have  lead  to  Conclusion 
4,  that  an  Automated  Message  Handling  System  must  be  an  integral  component  of 
a  communications  system  whose  overall  design  carefully  considers  the  ways  in 
which  the  users  will  interact  with  the  system.  The  total  system  must  be 
reliable;  it  must  provide  services  to  everyone  involved  with  message  handling 
in  the  user  environment;  it  must  provide  a  proper  balance  between  ephemeral 
displays  and  paper  copies. 
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SECTION  6 


CONCLUSIONS 


The  Military  Message  Experiment  is  an  effort  to  evaluate  the  utility  of 
computer-aided  message  handling  in  a  military  environment  and  to  determine  the 
type  of  automation  necessary.  The  Memorandum  of  Agreement  [reference  2] 
assigns  the  responsibility  for  evaluating  the  experiment  to  the  Navy.  Thus 
the  inputs  from  the  user  (CINCPAC)  and  the  developer  (DARPA)  are  used  by  the 
evaluator  (Navy)  in  evaluating  the  concept  of  automated  message  handling.  The 
user's  evaluation  of  the  system  is  included  in  Appendix  A.  It  was  one  of 
several  sources  used  in  developing  the  conclusions  in  this  section.  Other 
sources  include  on-site  observations,  analysis  of  user-requested  changes, 
interviews  with  users,  and  analysis  of  data  automatically  collected  by  the 
system. 

A  conscious  effort  has  been  made  to  distinguish  the  conclusions  that  are 
applicable  only  to  the  experimental  vehicle  (the  MME  system)  from  those  that 
are  applicable  to  the  concept  of  automated  message  handling.  Further,  because 
of  the  formal  and  informal  training  that  has  been  given  to  the  users,  some  of 
the  system  use  during  the  time  reported  on  may  have  been  in  response  to  that 
training.  For  instance,  a  set  of  proficiency  milestones  was  devised  by  the  J3 
staff  and  used  to  classify  the  proficiency  of  users.  There  is  no  way  to 
determine  from  the  statistics  recorded  whether  the  system  at  any  particular 
point  in  time  was  being  used  for  proficiency  training  or  to  accomplish  useful 
work.  Thus,  it  may  be  inappropriate  at  this  point  in  the  experiment  to 
conclude  that  the  most-used  MME  functions  are  the  most  important  functions  for 
a  future  message  processing  system.  But  there  are  certain  conclusions  that 
can  be  made  based  on  the  statistics  and  observations  of  the  users. 

The  following  are  conclusions  concerning  the  MME  system  rather  than  the 
concept  of  Automated  Message  Handling  Systems. 

Conclusion  1.  The  lack  of  reliability  of  the  MME  system  has  been  a 
major  complaint  of  the  CINCPAC  staff  users  and  has  had  an  impact  on 
their  perception  of  the  MME  system  in  particular  and  automated 
message  handling  in  general. 

Based  on  the  questionnaires  completed  by  the  MME  system  users  and  on 
interviews  conducted  with  fifteen  J3  division  and  branch  chiefs,  the  users 
generally  like  the  concept  of  an  Automated  Message  Handling  System  and  the  MME 
system  itself.  The  major  complaints  are  the  lack  of  reliability,  the  insuffi¬ 
cient  number  of  terminals,  and  the  slowness  of  the  MME  system  for  reading  and 
reviewing  messages.  These  complaints  were  factors  in  reaching  some  of  the 
more-general  conclusions  concerning  future  message  handling  systems. 
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Conclusion  2.  Despite  system  problems,  the  users  appear  to  prefer 
the  MME  system  to  the  manual  message-handling  system.  The  users  were 
asked  to  indicate  their  preference  for  automated  or  manual  message 
handling  in  each  of  three  general  categories:  receiving  messages, 
filing  and  retrieving  messages,  and  creating  and  sending  messages. 

(See  Appendix  A  for  a  more  detailed  analysis.)  The  daily  users 
indicated  a  slight  preference  for  automated  message  handling  for 
receiving  messages  and  a  distinct  preference  for  automated  message 
handling  for  filing,  retrieving,  creating,  and  sending  messages. 

Within  the  general  category  of  receiving  messages,  the  users  indicated 
a  distinct  preference  for  the  manual  system  for  the  initial  review  of 
incoming  messages.  The  exercise  users  indicated  a  slight  preference 
for  the  manual  system  for  receiving  messages,  a  distinct  preference 
for  the  automated  system  for  message  filing  and  retrieval,  and  a 
slight  preference  for  the  automated  system  for  creating  and  sending 
messages. 


The  "most-liked"  features  are  those  that  are  used  in  releasing  messages, 
editing  messages,  replying  to  received  messages,  creating  messages,  and 
retrieving  messages.  The  most-frequent  complaint  concerning  reviewing  incom¬ 
ing  messages  on  the  MME  system  is  that,  although  the  messages  are  received 
earlier  on  the  MME  system,  it  is  much  slower  than  the  manual  system  for 
actually  reading  or  scanning  the  message  text.  A  comparison  of  the  message 
delivery  times  in  the  automated  and  manual  systems  will  be  included  in  the 
final  report. 

The  goals  of  the  MME  include  evaluating  the  utility  and  the  cost- 
effectiveness  of  automated  message  handling  systems.  The  features  and 
performance  of  the  vehicle  (the  MME  system)  used  to  conduct  the  experiment  are 
important  in  the  determination  of  useful  features  for  future  systems.  In 
response  to  the  interviews  and  questionnaires,  the  users  have  been  very 
helpful  in  detailing  those  problems  in  the  MME  system  that  need  to  be  improved 
before  deployment  of  automated  message  handling  systems.  (See  Appendix  A.) 

In  writing  this  report,  the  evaluators  have  taken  care  to  separate  those 
comments  that  apply  only  to  the  experimental  vehicle  from  those  that  apply  to 
the  AMHS  concept.  Most  of  the  problem  areas  described  in  this  report  pertain 
to  the  MME  system  and  its  physical  environment  (including  power  and  communica¬ 
tions  links),  not  to  the  AMHS  concept.  Further,  the  cost  used  in  the  cost- 
effectiveness  evaluation  must  be  based  not  on  the  MME  system  cost,  but  on  the 
cost  of  the  technology  that  will  be  used  to  build  AMHSs  in  the  mid-to-late 
1980s. 

The  following  are  preliminary  conclusions  concerning  the  concept  of  User- 
Oriented  Automated  Message  Handling  Systems. 

Conclusion  3.  An  Automated  Message  Handling  System  is  useful  in  a 
military  environment. 

The  principal  goal  of  the  Military  Message  Experiment  is  to  determine  the 
utility  of  an  Automated  Message  Handling  System  in  a  military  staff 
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environment.  The  prelimiary  conclusion  is  that  such  a  system  is,  in  fact, 
useful  in  that  environment. 

A  reliable,  responsive  automated  message  handling  system  provides  message 
delivery  to  users  much  faster  than  the  manual  system.  A  system  that  properly 
integrates  the  MME  and  the  LDMX  functions  with  the  proper  mix  of  ephemeral  and 
hard-copy  displays  can  make  message  delivery  and  distribution  faster  and  more 
accurate.  An  AMHS  that  provides  a  coordination  and  release  function  that 
allows  users  to  interface  with  other  staff  members  either  by  paper  or  terminal 
can  speed  up  message  coordinations  while  maintaining  a  log  of  the  procedure. 
During  a  crisis,  a  command  staff  is  inundated  with  problems  that  must  be 
solved  quickly.  An  AMHS  can  aid  the  staff  by  providing  rapid  and  accurate 
message  filing  and  retrieval  and  aids  in  preparing  immediate  responses. 

Another  goal  of  the  experiment  is  to  develop  a  set  of  metrics  so  that  the 
usefulness  of  an  AMHS  can  be  measured  in  terms  of  the  Staff's  function.  Such 
measures  of  effectiveness  (MOE),  if  a  valid  set  can  be  developed,  would 
provide  a  basis  for  making  better  decisions  in  the  allocation  of  developmental 
resources.  The  effort  to  develop  the  measures  of  effectiveness  is 
continuing.  Data  are  being  collected  in  an  attempt  to  measure  delivery  and 
handling  times,  but  the  ultimate  value  of  speed  and  efficiency  are  extremely 
situation-dependent.  The  final  conclusions  and  recommendations  on  this  issue 
ultimately  must  be  based  on  analysis  of  both  the  quantitative  data  in 
conjunction  with  the  subjective  reactions  of  the  users. 

Conclusion  4.  An  Automated  Message  Handling  System  must  be  an 
integral  component  of  a  communications  system  whose  overall  design 
carefully  considers  the  ways  in  which  the  users  will  interact  with 
the  system.  The  total  system  must  be  reliable;  it  must  provide 
services  to  everyone  involved  with  message  handling  in  the  user 
environment;  it  must  provide  a  proper  balance  between  ephemeral 
displays  and  paper  copies. 

An  MME-type  system  will  be  only  one  component  of  the  future  communications 
system  that  serves  users  such  as  those  participating  in  this  experiment.  The 
MME  system  was  designed  to  provide  users  with  additional  message-handling 
features,  and  it  was  designed  to  be  a  compatible  component  within  an  existing 
communications  system.  Even  so,  one  of  the  more  frustrating  problems  for  the 
MME  system  users  has  been  the  lack  of  compatibility  of  the  MME  system  and  the 
rest  of  the  communications  system.  The  problems  are  cited  in  the  user's 
evaluation  in  appendix  A,  and  the  particular  problems  in  the  MME-LDMX 
interface  are  described  in  section  5. 

Thus,  one  of  the  MME's  important  lessons  for  future  system  designers  is 
that  an  integrated  design  for  the  total  communications  system  is  necessary  to 
ensure  user  satisfaction.  The  MME  evaluators  believe  that  separate  design  and 
development  responsibilities  for  user-oriented  message  processing  systems  and 
telecommunications  center  message  processing  systems  can  lead  to  serious 
component  compatibility  problems  such  as  those  that  exist  between  the  MME 
system  and  the  LDMX. 


Users  demand  that  an  interactive  system  that  they  depend  on  to  accomplish 
their  daily  tasks  and  to  resolve  problems  in  a  crisis  must  be  very  reliable. 
Although  they  may  not  notice  downtime  in  a  system  that  they  access  via  paper 
(such  as  the  LDMX) ,  they  are  acutely  aware  of  the  downtime  in  a  system  that  is 
accessed  via  a  terminal  in  their  office.  In  order  for  an  AMHS  to  be  reliable 
from  the  user's  perception,  its  software  must  be  thoroughly  tested  and 
debugged,  the  hardware  must  be  reliable  and  redundant,  and  the  system  must  be 
housed  in  a  non-hostile  environment.  The  experiment  has  demonstrated  that 
unreliable  power  can  have  a  disastrous  effect  on  system  reliability. 

The  need  to  provide  services  to  everyone  does  not  necessarily  mean  that 
everyone  involved  with  message  processing  must  have  a  terminal;  the  system  may 
recognize  that  some  users  are  serviced  by  hard-copy  output  and  by  a  central 
terminal  for  input.  But  the  system  must  have  well  thought-out  procedures  for 
including  those  users  in  message  distribution,  coordination,  and  release. 

There  are  some  tasks  that  users  can  do  more  efficiently  with  paper  copies  of 
messages.  There  are  other  tasks  where,  even  if  there  is  no  gain  in  efficiency, 
a  user  feels  more  comfortable  with  paper  copies.  It  will  be  crucial  in  future 
AMHSs  to  provide  an  adequate  number  of  displays,  an  adequate  number  of  printers 
strategically  located,  and  an  adequate  system  plan  for  providing  user  services 
via  paper  copies. 

Conclusion  5.  An  Automated  Message  Handling  5  -tern,  with  a  carefully 
designed  user  interface,  can  provide  a  useful  >jset  of  functions 
that  can  be  made  available  to  the  casual  user  without  the  need  for 
extensive  formal  training. 

For  those  who  will  be  heavy  users  of  an  AMHS,  there  should  be  formal 
training  so  that  they  interface  with  the  system  in  an  efficient  manner. 

Because  of  the  normal  turnover  in  a  military  staff,  there  will  always  be  new 
users  of  a  system.  The  user  interface  should  be  designed  to  accommodate  these 
new  users  and  casual  users  in  a  friendly  manner. 
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APPENDIX  A.  CINCPAC  USER  EVALUATION  OF  THE  SYSTEM 


I .  INTRODUCTION 

A.  Objective. 

The  primary  objective  of  the  MME,  as  defined  in  the  MOA,  is  "to  determine 
the  utility  of  an  interactive  message  service  in  a  major  military  head¬ 
quarters."  One  of  CINCPAC 's  responsibilities  under  the  MOA  is  to  participate 
in  this  evaluation.  The  objective  of  this  appendix  is  to  summarize  the  users' 
observations  and  evaluations  of  the  installed  equipment  and  software  and,  on 
that  basis,  to  identify  features  of  an  interactive  message  service  which,  even 
at  this  early  stage,  are  clearly  necessary,  and  to  identify  deficiencies  in 
the  current  system  which  impede  achievement  of  the  primary  objective  of  the 
program. 

B.  Outline  and  Summary. 

Sections  II  and  III  of  this  Appendix  describe  how  the  MME  was  used  and 
what  the  users'  observations  were  during  daily  operations  and  during  a  Command 
Post  Exercise  (CPX) ,  respectively.  These  observations  were  collected  by  means 
of  a  detailed  questionnaire  completed  by  23  daily  users  and  15  CPX 
participants,  supported  by  interviews  with  the  Deputy  J3,  a  Duty  Director  for 
Operations,  all  four  participating  division  chiefs,  and  eight  of  their  branch 
chiefs.  Although  the  users  believe  that  many  features  of  an  Automated  Message 
Handling  System  (AMHS)  would  be  valuable,  as  indicated  by  positive  overall 
ratings,  they  encountered  many  problems  with  the  lack  of  reliability,  scope, 
and  responsiveness  of  this  particular  implementation.  Section  IV  states  the 
current  conclusions  of  the  CINCPAC  staff  relative  to  the  objective  of  the 
experiment.  In  summary,  potential  manpower  savings  and  the  operational 
utility  of  an  AMHS  have  yet  to  be  demonstrated,  primarily  because  of  the 
problems  met  in  using  the  MME.  Section  V  presents  some  preliminary  recommen¬ 
dations  for  the  development  of  AMHSs,  emphasizing  the  need  to  correct  the  MME 
problems  and  achieve  interoperability  and  compatibility  with  other  message¬ 
handling  systems. 

II.  USE  IN  DAILY  OPERATIONS 
A.  Usage. 

1.  Starting  on  22  February  79,  with  the  beginning  of  Full  Experimental 
Use,  the  Operations  Directorate  attempted  to  use  the  MME  as  their  primary 
message-handling  system.  Participants  were  expected  to  use  the  MME  to  handle 
incoming  messages  (read  or  review  the  messages,  comment  on  them  or  on  the 
related  file  entries,  and  assign  action  and  forward  messages  to  other  users), 
to  file  and  retrieve  messages,  and  to  prepare  outgoing  messages  and  memoranda 
(create  the  message  or  memo,  coordinate  with  other  staff  offices,  edit  the 
in-preparation  message  or  memo,  and  release  it). 
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2.  During  the  period  22  February-7  April  1979,  preparation  of  messages 
and  memos  was  purposely  limited  because  the  system  had  not  demonstrated  that 
it  could  meet  the  requirements  for  full  operational  use  in  a  dynamic 
environment.  Its  initial  use  was  confined  to  those  actions  which  met  the 
following  criteria. 

(a)  The  security  classification  of  the  message  o-  memo  did  not  exceed 
that  of  “he  MME  system. 

(b)  There  were  at  least  two  days  of  working  time  available  to  meet 
the  assigned  suspense  date. 

(c)  All  items  referenced  by  the  message  or  memo  were  in  the  MME 
system's  data  base. 

(d)  All  offices  with  which  coordination  was  required  were  on  the 

system. 


3.  From  the  onset  of  MME  operations,  only  ACTION/COG  messages  have  been 
distributed  from  the  Directorate 's  Administrative  Branch  (J301)  to  the 
participating  divisions.  Participants  who  need  other  messages  have  access  to 
a  file  containing  all  messages  received  by  the  Operations  Directorate.  In  an 
attempt  to  evaluate  a  more  discrete  message  distribution  system,  one  division 
(J34)  established  a  selector  that  allowed  it  to  receive  an  approximation  of 
its  normal  distribution.  This  resulted  in  the  receipt  of  more  messages  than 
normal  because  the  selector  was  necessarily  broad  to  avoid  missing  any 
messages  of  interest.  Wider  use  of  similar  selectors  by  other  divisions  was 
not  practical  because  of  the  additional  administrative  burden  it  would  have 
placed  on  J301. 

4.  The  creation  of  Division  and  Branch  files  was  left  to  the  discretion 
of  the  participants.  After  the  start  of  FEU,  it  became  apparent  chat  most  of 
the  files  created  were  personal  files  for  individuals.  In  an  attempt  to 
evaluate  the  use  of  an  organizational  filing  system,  J34  crested  Division  and 
Branch  files  for  its  action  officers. 

B,  User  Observations. 

1.  During  this  period,  the  MME  system  had  a  significant  amount  of 
unscheduled  downtime  and  abnormal  terminations.  This  was  the  most  serious 
drawback  to  its  use.  The  users  evaluated  system  availability,  discounting 
scheduled  downtime,  as  unsatisfactory,  with  a  score  of  48  on  a  rating  scale 
of  0-100.  All  23  users  responded. 

2.  The  users  were  asked  to  compare  the  MME  system  to  the  manual  system  in 
each  of  12  message  handling  tasks.  The  score  of  100  shows  absolute  preference 
for  the  MME  system,  and  0  shows  absolute  preference  for  the  manual  system. 

The  parenthetical  entries  group  the  tasks  into  the  three  major  message-handling 
categories  and  show  weighted  averages  of  the  results.  The  use  factor  indicates 
the  percent  of  users  who  used  the  MME  system  for  that  particular  task. 
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TASK/CATEGORY 

PREFERENCE 

USE  FACTOR  (%) 

Reviewing  Incoming  Messages 

34 

91 

Commenting  on  Incoming  Messages 

58 

43 

Assigning  Action 

61 

48 

Passing  Messages  to  Others 

71 

74 

(Incoming  Messages) 

(54) 

Filing  Messages 

64 

83 

Maintaining  Files 

65 

83 

Retrieving  Messages 

75 

96 

(Filing/Retrieval) 

(68) 

Creating  Messages 

78 

83 

Replying  to  Messages 

77 

74 

Coordinating  In-prep  Messages 

74 

65 

Editing  In-prep  Messages 

77 

78 

Releasing  messages 

79 

65 

(Outgoing  Messages) 

(77) 

3.  Users  evaluated  the  responsiveness  of  the  MME  system  for  the  same  12 
message-handling  tasks,  using  the  rating  system  described  in  paragraph  B.2. 

The  differences  between  the  Use  Factors  shown  in  this  table  and  those  shown  in 
the  table  above  result  from  inconsistent  responses  by  a  few  of  the  respondents. 


TASK/CATEGORY 

SATISFACTION 

USE  FACTOR  (%) 

Reviewing  Incoming  Messages 

60 

100 

Commenting  on  Incoming  Messages 

64 

57 

Assigning  Action 

70 

48 

Passing  Messages  to  Others 

79 

70 

(Incoming  Messages) 

(67) 

Filing  Messages 

79 

91 

Maintaining  Files 

75 

87 

Retrieving  Messages 

76 

100 

(Filing/Retrieval) 

(76) 

Creating  Messages 

81 

96 

Replying  to  Messages 

81 

65 

Coordingating  Messages 

74 

57 

Editing  In-Prep  Messages 

71 

70 

Releasing  Messages 

71 

52 

(Outgoing  Me-sages 

(76) 

4.  Users'  comments  on  the  utility  of  the  MME  in  performing  the  12 
message-handling  tasks  are  summarized  below. 

(a)  Incoming  messages.  Handling  incoming  messages  received  a 
preference  rating  of  54  and  a  responsiveness  rating  of  67  [indicating  a  slight 
preference  for  the  use  of  the  MME  system  and  a  stronger  preference  for  the 
responsiveness  of  the  MME  system  over  the  manual  system] ,  but  most  user 
comments  were  negative. 
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(1)  Reviewing  incoming  messages  on  the  MME  system  is  too  time 
consuming.  The  file  citations  in  many  cases  do  not  show  enough  information  to 
determine  whether  or  not  to  call  up  and  read  the  message.  To  be  certain,  the 
message  in  question  must  be  viewed  or  displayed,  which  takes  much  more  time 
per  message  than  with  the  manual  system.  The  scrolling  process  can  be  very 
slow,  especially  with  a  high  load  average.  Most  users  said  they  can  scan  a 
paper  message  much  more  quickly  than  a  message  on  the  MME  screen,  and  that 
they  can  go  back  and  forth  on  a  paper  message  much  more  rapidly  than  on  the 
MME  image.  Having  to  scroll  through  a  long  list  of  addressees  prior  to 
reaching  the  message  text  also  wastes  time.  The  inability  of  the  MME  system 
to  group  multi-sectioned  messages  is  a  drawback.  The  relatively  small  number 
of  terminals  allows  only  one  person  at  a  time  in  each  branch  to  look  at 
incoming  messages  on  the  MME  system.  Some  users  found  the  message  review 
process  better  on  the  MME  system  because  they  could  look  at  all  the  incoming 
messages  to  J3  (within  the  limits  of  the  security  constraints)  rather  than 
getting  only  the  selected  messages  delivered  to  them  by  the  manual  system. 

(2)  Annotating  incoming  messages  using  the  MME  system  is 
awkward,  slow,  and  limited  compared  to  the  paper  system.  Moreover,  the 
comments  can't  always  be  positioned  where  the  user  really  wants  them.  On  the 
paper  copy,  it  is  much  easier  to  make  a  note  in  the  margin,  or  to  underline  or 
highlight  portions  of  the  text.  However,  comments  made  on  the  MME  system 
stand  out  very  distinctly. 

(3)  Assigning  action  on  a  message  is  easier  in  the  manual 
system  by  stamping  or  writing  "Act  -  Jxxx"  on  the  paper  copy.  Because  of  this 
ease  and  the  additional  administrative  burden  of  operating  two  suspense 
systems,  the  assignment  of  actions  below  the  directorate  level  was  not 
exercised.  Not  having  all  of  J3  on  the  MME  system  limits  the  benefits  of  this 
MME  system  function. 

(A)  The  forwarding  capability  of  the  MME  system  is  potentially 
very  useful.  Electronic  transmission  of  messages  is  much  faster  than  in  the 
manual  system.  Again,  not  having  all  of  J3  or  any  other  directorate  on  the 
MME  system  limits  the  usefulness  of  this  feature.  Moreover,  if  the  recipient 
is  not  at  his  terminal,  the  advantage  of  the  transmission  speed  is  lost. 

(5)  The  system  caused  a  shift  in  some  administrative  respon¬ 
sibilities  to  the  action  officers.  With  the  message  distribution  system  used, 
the  individual  action  officers  had  to  "pull"  messages  from  the  daily  message 
file  instead  of  having  messages  of  interest  "pushed"  to  them  by  administrative 
personnel. 


(6)  J34  and  its  branches  (J341  and  J342)  created  a  set  of  files 

that  are  used  for  processing  incoming  message  traffic  as  it  is  received  from 
J301.  Each  morning  the  Branch  Chiefs  review  the  J34  pending  file,  delete 
non-germane  messages,  and  use  the  remaining  messages  to  brief  the  Division 
Chief  on  items  of  immediate  interest  prior  to  his  daily  meeting  with  the  J3. 
The  deficiencies  noted  in  paragraphs  (1)  and  (2)  above  severely  limit  the 
usefulness  of  the  MME  system  in  this  process.  When  the  Division  Chief  has 
completed  his  message  review,  the  Branch  Chiefs  route  the  remaining  messages 
to  various  files  for  review  by  the  action  officers.  The  J34  users  found  that 


33 


by  the  time  the  messages  had  been  routed  all  the  way  to  the  branch  files  and 
the  users  had  been  able  to  get  access  to  the  one  available  terminal,  they 
could  have  already  seen  the  messages  in  the  paper  system  and  started  any 
required  actions. 

(b)  Message  filing  and  retrieval.  The  users  found  the  MME  system 
better  for  message  filing  and  retrieval.  The  following  advantages  and 
disadvantages  of  the  MME  system  were  cited. 

(1)  Most  users  found  message  filing  in  personal  files  much 
faster  and  handier  on  the  MME  system. 

(2)  Branch  and  division  files  were  not  generated  as  a  result  of 
daily  operational  use.  The  files  created  specifically  for  evaluation  purpose, 
as  noted  in  paragraph  A. 4  above,  were  not  used,  primarily  because  action 
officers  who  had  carefully  maintained  their  paper  files  could  retrieve  a 
message  faster  from  them.  An  important  secondary  reason  was  the  unavailability 
of  files  because  of  system  unreliability. 

(3)  In  some  special  applications,  maintaining  files  on  the  MME 
system  is  handier  and  faster  than  on  the  manual  system.  Common-user  files  are 
easier  to  maintain  manually.  The  slowness  of  message  review  makes  selective 
purging  of  MME  files  too  time  consuming. 

(4)  Retrieval  of  messages  from  MME  files  was  found  to  be  a  very 
valuable  capability  for  messages  which  the  user  had  not  kept  in  well-organized 
paper  files.  The  ability  of  the  MME  system  to  retrieve  messages  on  arguments 
other  than  DTG  and  on  multiple  arguments  allowed  flexibility  in  the  search  for 
messages  which  could  not  be  identified  definitively.  It  was  also  beneficial 
in  retrieving  complete  categories  of  messages.  The  principal  complaint  about 
message  retrieval  was  the  length  of  time  required  to  get  messages  from  the 
archives.  Also,  if  the  exact  DTG  of  a  message  is  not  known,  a  search  must  be 
made  through  several  datefiles  in  order  to  retrieve  the  message  by  other 
arguments  (subject,  originator,  etc). 

(c)  Outgoing  messages.  Although  processing  of  outgoing  messages 
received  a  preference  rating  of  77  and  a  responsiveness  rating  of  76,  the 
tenor  of  the  user  comments  was  negative. 

(1)  A  lack  of  terminal  availability  hampered  use  of  the  MME 
system  in  offices  with  large  numbers  of  outgoing  messages  and  memos.  The  MME 
system  tends  to  turn  action  officers  into  clerk/typists,  with  the  requirement 

to  learn  message  formats,  administrative  procedures,  etc. 

1 

(2)  The  coordination  cycle  was  poor  because  it  was  cumbersome 
to  use  and  did  not  model  the  real-world  process  well.  Unless  all  coordinators 
were  alerted  and  were  awaiting  the  arrival  of  a  message  or  memo  for  chop,  the 
coordination  cycle  normally  took  longer  on  the  MME  system  than  manually.  Users 
said  that  face-to-face  coordination  on  many  messages,  especially  those  that 
raise  a  controversial  issue,  is  much  more  effective.  Coordination  was  also 
harapered  because  sometimes  not  all  references  for  an  action  are  on  the  system. 
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(3)  Users  found  that  editing  or  consnenting  on  in-preparation 
messages  or  memos  is  just  as  easy  in  the  manual  system  as  in  the  MME  system, 
especially  if  the  typist  has  a  word-processing  facility. 

(4)  Users  foresee  an  excellent  potential  for  the  release  of 
messages  by  the  MME  system  into  the  AUTODIN  system,  but  problems  in  the 
Sigma-Local  Digital  Message  Exchange  (LDMX)  interface  [see  Section  S  of  this 
report]  have  caused  a  reluctance  to  trust  the  MME  system.  Users  were  unanimous 
in  their  enthusiasm  for  the  MME  system's  readdressal  capability.  Distribution 
of  memos  to  participating  J3  divisions  was  done  on  a  regular  basis  without  any 
system  problems.  External  distribution  caused  awkward  control  and  routing 
problems  at  the  electronic/manual  interface. 

III.  EXERCISE  USE. 

A.  General. 

1.  The  MME  system  was  used  to  support  the  CINCPAC  Operations  Action  Group 
during  Exercise  Power  Play  79  (PP-79),  a  JCS  world-wide  CPX  conducted  6-23 
March  1979.  This  section  discusses  the  composition  and  functions  of  the 
Operations  Action  Group,  the  manner  in  which  the  MME  system  was  used  to 
support  it,  the  highlights  and  problems  encountered,  and  observations  of  the 
MME  system's  usefulness  in  supporting  Operations  Action  Group  message-handling 
tasks. 

B.  OAG  Composition  and  Functions. 

1.  The  OAG  is  a  small  staff  element  formed  to  support  the  CINCPAC  Crisis 
Action  Coordinator  during  a  crisis  or  exercise.  It  handles  all  crisis  or 
exercise  matters  requiring  immediate  attention,  including  handling  of  incoming 
and  outgoing  message  traffic. 

2.  The  OAG  normally  consists  of  the  Deputy  Crisis  Action  Coordinator 
(DCAC),  Chief  of  the  OAG  (COAG) ,  Executive  Officer  (XO) ,  Assistant  Executive 
Officer  (AXO),  and  an  action  officer  from  each  of  the  CINCPAC  staff  sections 
(Jl,  J2,  J3,  J4,  J5,  J6,  and  J74  (Public  Affairs)).  Due  to  the  planned  low 
level  of  CINCPAC  activity  in  PP-79,  an  abbreviated  OAG  was  formed,  consisting 
of  the  DCAC,  XO,  and  J2,  J3,  J4,  J5  and  J6  action  officers.  In  addition,  an 
officer  was  assigned  to  use  the  MME  to  parallel  and  duplicate  the  duties  of 
the  OAG's  XO  and  AXO.  This  officer  was  known  as  the  MXO  (MME  XO) . 

3.  The  XO  and  AXO  are  the  key  members  in  the  manual  handling  of  OAG 
messages.  The  XO  receives  all  incoming  traffic;  assigns  action  or  cognizance 
to  the  appropriate  OAG  action  officer;  keeps  the  DCAC  or  COAG  informed  of 
significant  messages;  maintains  a  Significant  Events  Log  (SEL);  ensures  that 
all  OAG  actions  have  been  properly  coordinated;  and  reviews  outgoing  messages 
for  proper  format,  coordination,  and  content  prior  to  release  by  the  DCAC  or 
COAG.  The  AXO  maintains  the  master  message  files,  a  log  of  incoming  and 
outgoing  messages,  and  a  Status  of  Action  Log  (SAL),  in  which  he  records 
pertinent  data  on  OAG  actions  (who  is  responsible  for  the  action,  suspense 
date,  time  of  completion,  etc). 
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C.  MME  Support  for  the  OAG. 

1.  The  MME  system  was  a  backup  for  the  manual  system  for  all  OAG 
actions.  This  was  done  to  ensure  that  failure  of  the  MME  system  would  not 
interfere  with  the  timely  completion  of  OAG  actions. 

2.  Only  the  MXO  had  a  dedicated  MME  system  terminal.  The  DCAC  and  other 
OAG  AOs  used  three  additional  terminals  as  required.  A  fifth  terminal  was 
available  but  was  never  used. 

3.  The  MXO  did  not  work  full  time  throughout  the  exercise  because  the 
volume  of  message  traffic  was  low.  However,  in  the  MXO's  absence,  the  XO  in 
many  cases  processed  incoming  messages  on  the  MME  system  in  addition  to  his 
manual  message  processing,  because  he  had  time  available. 

4.  Specific  OAG  automated  message  handling  procedures. 

(a)  MXO.  All  incoming  messages  on  the  MME  came  to  the  OAG  Pending 
File  (OAG  MXO).  The  MXO  reviewed  each  incoming  message  citation  (or  message, 
if  necessary).  Lf  there  were  an  OAG  action  required,  he  assigned  the  message 
to  the  appropriate  OAG  action  officer  (AO)  and  updated  the  SAL.  If  the 
message  were  a  COG  item,  he  forwarded  it  to  the  appropriate  OAG  AO.  All 
incoming  messages  were  filed  in  the  OAG  Master  file.  Important  messages  were 
forwarded  to  the  DCAC.  The  MXO  maintained  and  updated  the  SEL  as  required. 

The  MXO  reviewed  all  outgoing  messages,  ensured  all  chops  had  been  made, 
resolved  any  non-concurrences,  and  forwarded  messages  to  the  DCAC  for 
release.  If  the  outgoing  message  satisfied  a  pending  OAG  action,  the  MXO 
updated  the  SAL.  If  the  outgoing  message  directed  a  subordinate  to  act  or 
requested  action  from  a  co-lateral  or  higher  headquarters,  the  MXO  posted  the 
appropriate  files.  All  outgoing  OAG  messages  were  filed  from  the  OAG  Pending 
File  to  one  of  the  three  outgoing  files.  The  MXO  ensured  that  all  actions 
generated  from  non-message  sources  were  entered  into  the  SAL  and  SEL  as 
appropriate.  The  MXO  readdressed  messages  as  directed  by  the  DCAC. 

(b)  DCAC.  A  terminal  was  available  for  the  DCAC,  but  was  rarely 
used.  Releasing  outgoing  messages,  especially  readdressals ,  was  the  DCAC 's 
primary  use  of  the  system. 

(c)  Action  Officers.  The  AOs  had  two  terminals  available  for  their 
exclusive  use,  in  addition  to  being  able  to  use  the  DCAC's  terminal  most  of 
the  time.  Many  AOs  made  extensive  use  of  the  MME  system  during  the  time 
available  to  them  when  they  were  not  processing  actions  using  the  manual 
system.  The  J2  and  J4  AOs  created  extensive  files  to  support  the  wide  variety 
of  messages  received  during  the  exercise. 

D.  User  Observations  During  Exercise  Power  Play  79. 

1.  The  situation  in  which  the  MME  system  was  used  for  crisis  or  exercise 
support  during  PP-79  was  unusual  in  that  the  OAG  was  required  to  process 
relatively  few  outgoing  messages  (in  comparison  to  the  Nifty  Nugget  exercise 
conducted  in  the  fall  of  1978).  Thus  there  was  only  a  limited  opportunity  to 
exercise  the  message  creation,  coordination,  editing,  and  release  cycle  using 
the  MME  system. 
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2.  Fifteen  OAG  users  were  asked  to  complete  a  detailed  questionnaire 
concerning  the  operational  usefulness  of  the  MME  system  to  them  during  PP-79. 
The  following  is  a  summary  of  their  evaluation. 

(a)  The  users  were  asked  to  compare  the  MME  system  to  the  manual 
system  in  each  of  12  message-handling  tasks,  using  the  rating  system  described 
in  Section  II,  paragraph  B.2. 


TASK/CATEGORY 

PREFERENCE 

USE  FACTOR 

Reviewing  Incoming  Messages 

40 

100 

Commenting  on  Incoming 

37 

80 

Assigning  Action 

48 

53 

Passing  Messages  to  Others 

58 

80 

(Incoming  Messages) 

(47) 

Filing  Messages 

74 

100 

Maintaining  Files 

73 

100 

Retrieving  Messages 

84 

93 

(Filing/Retrieval) 

(77) 

Creating  Messages 

58 

93 

Replying  to  Messages 

46 

67 

Coordinating  In-Prep  Messages 

41 

73 

Editing  In-Prep 

66 

80 

Releasing  Messages 

66 

67 

(Outgoing  Messages) 

(55) 

(b)  Users  were  asked  to  evaluate  the  responsiveness  of  the  MME  system 
for  the  same  12  message-handling  tasks  using  the  same  rating  scheme.  Again, 
the  difference  in  the  use  factor  is  due  to  some  inconsistent  user  responses. 


TASK/CATEGORY 

SATISFACTION 

USE  FACTOR  (%) 

Reviewing  Incoming  Messages 

54 

93 

Commenting  on  Incoming 

54 

80 

Assigning  Action 

64 

53 

Passing  Messages  to  Others 

63 

67 

(Incoming  Messages) 

(58) 

Filing  Messages 

77 

100 

Maintaining  Files 

70 

93 

Retrieving  Messages 

76 

100 

(Filing/Retrieval) 

(75) 

86 

Creating  Messages 

61 

Replying  to  Messages 

59 

60 

Coordinating  Messages 

55 

73 

Editing  -  In-Prep  Messages 

62 

80 

Releasing  Messages 

70 

46 

(Outgoing  Messages) 

(61) 

(c)  The  users  evaluated 

system  availability,  discounting  scheduled 

downtime,  as  unsatisfactory,  with 

a  score  of  45  on  a  rating 

scale  of  0-100. 

All  fifteen  users  responded. 
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(d)  In  general,  the  written  coaments  of  the  users  paralleled  their 
evaluation  in  the  numerical  portion  of  the  questionnaire.  They  showed  a 
preference  for  the  manual  review  of  incoming  messages,  a  preference  for  the 
MME  system's  handling  of  message  filing  and  retrieval,  an  indifference  between 
the  systems  for  preparation  of  outgoing  messages,  dissatisfaction  with  MME 
system  availability,  and  reluctance  to  rely  on  the  MME  system  as  the  sole  or 
primary  message-handling  system. 

3.  Highlights  of  MME  System  Use. 

(a)  The  MME  system,  when  available,  was  able  to  process  and 
distribute  messages  in  a  timely  manner.  However,  see  comments  in  paragraphs 
4(b)  and  4(f)  below. 

(b)  The  readdressal  feature  was  enthusiastically  received  by  the  OAG, 
AOs  and,  once  demonstrated,  was  used  for  almost  all  message  readdressals. 

(c)  During  periods  of  low  OAG  activity,  some  outgoing  messages  were 
created  and  released  on  the  MME  system.  On  one  occasion,  the  CINCPAC  S1TREP 
was  prepared  on  various  AO  terminals,  assembled  by  the  MXO  on  his  terminal  and 
released  by  the  DCAC  on  his  terminal. 

(d)  The  ability  to  retrieve  messages  from  files  was  the  system 
feature  for  which  the  users  expressed  the  highest  degree  of  preference. 

4.  Problems  Encountered. 

(a)  The  major  problem  with  the  use  of  the  MME  system  was  the 
unacceptable  amount  of  unscheduled  system  downtime.  The  lack  of  system 
reliability  mandates  a  backup  or  redundant  message-message  handling  system. 

(b)  Initially  a  very  high  load  average  caused  by  a  software  failure 
resulted  in  lengthy  delays  in  executing  commands.  Subsequently,  an  additional 
22 1  of  CPU  time  was  allocated  to  the  OAG  user  group.  This  increased  processing 
speed  to  an  acceptable  level,  but  at  the  expense  of  the  remaining  J3  users, 
who  were  reduced  to  10%  in  toto.  Delays  also  occurred  during  the  coordination 
or  release  process.  Once  the  message  left  the  OAG  user  job,  it  lost  the 
advantage  of  the  OAG's  CPU  slice.  On  one  occasion,  it  took  over  an  hour  for  a 
message  to  get  from  one  OAG  terminal  through  the  processing  cycle  and  arrive 

at  another  OAG  terminal. 

(c)  The  non-J3  members  of  the  OAG  were  inexperienced  in  the  use  of 
the  MME  system  and  the  terminal.  Although  a  training  program  had  been 
established  in  January  1979  specifically  for  non-J3  OAG  members,  most  of  the 
non-J3  members  who  actually  participated  in  the  exercise  were  not  fully 
qualified  on  the  system.  This  was  overcome  by  intensive  OJT.  Within  a  few 
sessions  the  non-J3  OAG  members  were  capable  of  using  the  system  to  display, 
file,  retrieve,  create,  and  comment  on  messages  with  minimal  assistance. 
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(d)  The  precedence  of  exercise  or  crisis  messages  will  normally  be 
Immediate  or  Flash.  Currently,  the  MME  system  does  not  distinguish  message 
precedences  in  assigning  processing  priority,  so  that  a  high  precedence 
message  may  be  delayed  in  a  queue  while  lower  precedence  messages  are  being 
processed. 

(e)  The  release  of  OAG-originated  and  readdressed  messages  was 
hampered  by  previously  unknown  interface  problems  between  Sigma  and  the  LDMX. 

A  related  problem  was  the  lack  of  notification  to  the  user  when  a  message  was 
rejected  by  the  LDMX. 

(f)  The  MXO,  who  was  fully  trained  and  experienced  on  the  MME  system, 

was  totally  occupied  in  trying  to  accomplish  all  the  tasks  he  was  supposed  to 
perform  during  the  periods  he  was  working  on  the  MME  system.  At  times  he  was 
not  able  to  keep  up  with  every  task,  and  had  to  postpone  some,  such  as 
maintaining  the  SAL  or  other  files  containing  suspense  items.  The  MXO  was 
able  to  catch  up  with  incoming  message  distribution  in  approximately  one  hour 
after  being  absent  for  an  extended  period  (up  to  8-10  hours),  due  to  the  low 
message  arrival  rate.  Since  there  were  relatively  few  outgoing  messages 
processed  on  the  MME  system,  the  MXO  did  not  spend  much  time  on  this  facet  of 
message  handling.  The  MXO  did  spend  some  time  assisting  the  other  OAG  users. 
This  would  not  be  required  if  all  OAG  members  were  fully  trained.  The  two 
MXOs  for  the  exercise  agreed  that  one  MXO  per  12-hour  shift  would  not  have 

been  able  to  handle  all  the  tasks  assigned  to  the  MXO  for  PP-79  if  the 

incoming  message  flow  had  been  comparable  to  that  of  Exercise  Nifty  Nugget. 

For  an  exercise  of  that  scale  it  would  have  taken  at  least  two  MXOs  per  shift 
to  handle  all  desired  functions  on  the  MME  system.  Although  this  does  not 
represent  any  manpower  savings  over  the  manual  system,  the  use  of  the  MME 
system  should  be  of  great  assistance  to  the  MXOs  in  several  of  their 
message-handling  tasks  (providing  the  system's  reliability  and  responsiveness 
were  satisfactory).  The  MME  system's  speed  and  filing  system  permit  much  more 
efficient  maintenance  of  the  master  message  file  and  log,  the  SAL,  and  the  SEL. 

(g)  The  ability  of  an  MXO  or  AO  to  remain  at  the  terminal  and 

function  efficiently  for  a  long  shift  on  a  continuing  basis  during  a  crisis  or 

exercise  is  doubtful.  After  a  period  of  time  the  ability  to  concentrate  on 
performing  various  operations  or  to  look  at  the  screen  without  its  becoming 
blurred  is  lost.  In  PP-79,  the  message  inflow  was  low  enough  that  the  MXO 
periodically  could  leave  the  terminal  without  falling  irretrievably  behind. 
Taking  a  rest  period  during  Nifty  Nugget  could  easily  have  resulted  in  a  long, 
arduous  session  at  the  terminal  to  catch  up.  It  is  possible  that  one  of  the 
AOs  could  spell  the  MXO  from  time  to  time,  but  during  Nifty  Nugget  most  AOs 
were  busy  a  great  part  of  the  time  on  their  own  work.  Frequent  relief  for 
high-use  terminal  operators  definitely  must  be  considered  in  planning  the 
watch  bill  for  an  exercise  or  crisis. 

5.  During  one  portion  of  the  exercise,  a  record  of  message  arrival  times 
was  maintained  by  the  MXO.  High  precedence  messages  arrive  at  the  OAG  on  a 
line  printer  directly  from  the  LDMX.  Lower  precedence  messages  and  back-up 
copies  of  high  precedence  messages  arrive  at  the  OAG  via  pneumatic  tube  from 
Che  CINCPAC  Communications  Center.  The  MXO  kept  a  log  of  incoming  message 
time  of  receipt  (TOR),  matching  the  TOR  of  the  high-precedence  messages  on  the 
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line  printer  versus  their  TOR  on  the  MXO's  terminal.  Of  the  37  messages 
recorded  on  the  MXO's  TOR  log,  the  average  TOR  on  the  MME  system  was  3.4 
minutes  later  than  the  TOR  on  the  line  printer.  The  TOR  on  the  MME  system  was 
consistently  well  ahead  of  the  arrival  of  the  backup,  hard-copy  messages 
received  through  the  pneumatic  tube  from  the  Communications  Center.  The 
differences  varied  from  a  low  of  two  minutes  to  a  high  of  29  minutes. 

IV.  CINCPAC  CONCLUSIONS 

A.  MME  Contributions  to  Efficiency  and  Manpower  Savings.  Each  of  the 
officers  interviewed  during  the  data  collection  phase  of  the  J3  evaluation  was 
asked  to  comment  on  the  perceived  increase  in  efficiency  of  his  office's 
operation  due  to  the  use  of  the  MME  system  and  the  manpower  savings  foreseen 
by  use  of  the  MME  system  in  an  operational  mode. 

1.  Most  of  the  officers  saw  no  increase  in  message-handling  efficiency 
as  a  result  of  using  the  MME  system.  This  was  primarily  due  to  low  system 
reliability,  the  ability  to  process  incoming  messages  faster  using  the  manual 
system,  and  an  inadequate  distribution  of  terminals. 

2.  Most  of  the  officers  did  not  feel  that  use  of  the  MME  equipment  as  an 
operational  system  would  allow  manpower  savings  in  their  offices,  although  the 
possibility  of  realizing  some  savings  if  the  system  were  to  be  expanded 
J3-wide  or  CINCPAC-wide  was  recognized.  However,  these  savings  might  be 
offset  in  part  by  the  manpower  required  to  operate  and  maintain  the  MME  system. 

B.  Operational  Usefulness  of  an  AMHS.  The  operational  usefulness  of  an  AMHS 
in  a  major  military  headquarters  has  not  yet  been  demonstrated.  While  the  MME 
system  has  a  number  of  features  that  are  very  beneficial  for  both  daily  and 
crisis  or  exercise  use,  it  has  not  been  able  to  perform  as  a  complete  system 
in  a  way  that  would  permit  a  definitive  evaluation.  The  following  factors 
have  prevented  such  an  evaluation. 

1.  The  reliability  of  the  MME  system  has  not  been  sufficient  to  permit  it 
to  be  used  as  the  primary  message-handling  system.  System  downtime  and 
abnormal  terminations  have  plagued  the  experiment  from  the  beginning.  Without 
a  reliable  testbed,  the  users  must  continue  to  rely  on  the  manual  system  for 
most  of  their  message  handling. 

2.  There  is  an  insufficient  number  of  terminals  to  support  the  divisions 
now  participating  in  the  MME.  J3  users  must  compete,  in  many  instances,  for 
the  limited  number  of  terminals  supporting  their  branch.  This  leads  to  many 
actions  being  conducted  using  the  manual  system,  due  to  time  constraints  and 
other  factors,  and  inhibits  the  user  interface  with  the  system.  Not  having 
all  of  the  Operations  Directorate  or  any  other  CINCPAC  directorate  on  the  MME 
system  causes  inefficiencies  in  the  message-handling  scheme,  e.g.,  outgoing 
messages  must  be  hand-carried  to  non-users  for  coordination.  Although  the 
current  MME  hardware  can  accept  58  user  input-output  devices,  this  would  not 
significantly  resolve  the  terminal  problem.  The  benchmark  testing  has  shown 
chat  although  the  system  can  handle  20  users,  at  that  point  the  response  time 
begins  to  slow  markedly.  An  attempt 
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Co  service  all  58  terminals  could  degrade  system  response  time  to  a  level 
completely  unacceptable  to  the  users.  The  current  system  architecture  is  such 
that  it  cannot  support  the  number  of  users  needed  to  demonstrate  that  an  AMHS 
is  operationally  useful  at  CINCPAC  for  general  staff  support.  It  may,  however, 
be  sufficient  to  demonstrate  utility  for  crisis  operations  if  it  is  used  in  a 
major  CPX. 

3.  The  man/machine  interface  became  saturated  at  relatively  low  message 
volumes  in  crisis  situations.  Extended  use  of  the  terminal  also  causes 
interface  problems.  Because  opportunity  to  stress  the  system  in  a  high 
message  volume  situation  has  not  occurred,  the  ability  of  the  system  to 
support  a  crisis  remains  undemonstrated. 

V.  CINCPAC  RECOMMENDATIONS. 

A.  No  AMHS  should  be  developed  or  fielded  without  the  most  intimate 
participation  of  the  using  community.  The  potential  users  of  an  AMHS  must  be 
closely  involved  in  the  development  of  system  features  and  operating 
characteristics.  Failure  to  have  the  operation  of  the  proposed  system 
rigorously  checked  in  great  detail  by  the  prospective  users  will  result  in  a 
system  that  is  an  inadequate  approximation  of  the  one  required.  The  structure 
of  an  AMHS  must  be  designed  to  facilitate  incorporation  of  or  deletion  of 
specific  operational  capabilities  without  disruption  of  the  overall  system 
model.  An  AMHS  should  be  given  to  a  user  community  only  after  through  testing 
involving  selected  users.  The  MME  has  been  a  start  in  the  right  direction,  but 
user  involvement  during  the  experiment  should  be  maintained,  day-to-day, 
during  the  development  of  an  operational  system. 

B.  J3  users  attempted  to  use  the  MME  system  as  a  substitute  for  existing 
manual  message-handling  procedures.  While  this  approach  was  mandated  by  the 
requirement  to  keep  the  manual  system  operational,  in  many  cases  it  precluded 
devising  operational  procedures  which  would  have  better  used  the 
characteristics  of  the  automated  system.  Manual  and  automated  procedures  are 
not  interchangeable  if  either  system  is  to  be  optimized.  Consequently,  the 
design  of  the  AMHS  of  the  future  should  not  be  constrained  by  the  using 
organization's  current  operating  procedures. 

C.  Reliability  is  the  single,  most  important  design  criterion  for  an  AMHS. 

No  system  should  be  fielded  until  it  can  demonstrate  reliability  at  least 
equal  to  current  message  processing  systems  such  as  the  LDMX.  Adequate  backup 
systems  and  procedures  must  be  provided  and  maintained,  not  only  to  ensure 
that  messages  can  be  sent  and  received,  but  also  to  allow  access  to  files  and 
archives. 

D.  The  system  must  be  broadly  based.  The  limited  network  of  terminals 
available  in  the  J3  Directorate  reduced  the  benefits  available  from  automated 
message  handling.  Future  systems  should  provide  a  dedicated  terminal  for  each 
management  level  person,  crisis  action  team  member,  and  administrative 
person.  A  terminal  density  of  one  per  two  action  officers  appears  adequate 
for  daily  use,  but  should  be  tailored  to  specific  user  requirements. 
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G.  The  system  must  be  responsive.  The  MME  system  is  too  slow,  both  in 
executing  certain  commands  and  in  the  scrolling  process.  Two  seconds  is  a 
desirable  time  for  the  system  to  respond  to  a  user  input.  System  response  is 
also  slowed  by  procedural  requirements  for  entering  commands,  confirming 
commands,  checking  security,  logging  on,  etc.  Additional  function  keys  would 
alleviate  some  of  the  problem  if  the  response  time  could  be  minimized. 

F.  The  system  must  be  compatible  with  other  means  of  message  handling. 

1.  Currently  the  only  method  for  entering  user-originated  messages  or 
memos  is  through  the  keyboard.  Expansion  of  system  input  devices  to  include 
an  Optical  Character  Reader  or  a  reader  of  magnetic  cards  would  increase  the 
system's  flexibility.  Once  the  message  or  memo  is  entered,  the  action  officer 
can  make  any  editorial  changes,  using  the  MME  system,  with  relatively  little 
effort. 


2.  The  system  must  be  compatible  with  a  procedure  for  handling  material 
that  cannot  be  accomodated  on  the  terminal.  Maps,  correspondence,  items  with 
higher  security  classif ications ,  and  bound  documents  are  examples  of  material 
that  must  sometimes  accompany  messages  and  memos  through  the  coordination 
process. 

G.  The  system  must  be  interoperable  with  its  supporting  message  transmission 
system.  The  problems  encountered  in  the  Sigma-LDMX  interface  have  caused  a 
great  deal  of  user  dissatisfaction. 

H.  The  current  military  message  standards  contained  in  Joint  Army/Navy /Air 
Force  Procedure  (JANAP)  128  are  not  specific  enough  to  allow  the  most 
efficient  automated  processing  of  military  messages.  The  current  standard  for 
the  subject  line  of  a  message  is  overly  broad,  making  the  creation  of  an 
effective  algorithm  for  determining  the  message  subject  difficult.  In  many 
cases  this  results  in  the  citation  displaying  message  text  in  place  of  the 
subject.  JANAP  128  should  be  revised  to  require  a  more  standardized  military 
message  format,  especially  with  regard  to  identification  of  the  subject. 

I.  Processing  priority  based  on  message  precedence  is  required  to  ensure  fast 
delivery  of  high  precedence  messages,  both  incoming  and  in-preparation  or 
released. 

J.  While  we  have  demonstrated  that  an  AMHS  can  perform  certain  functions  as 
expected,  it  is  unclear  that  significant  benefits  will  be  realized  from  the 
automation  process.  The  perceived  lack  of  increased  efficiency  of  the  MME 
system  by  the  users  indicates  that  a  thorough  evaluation  of  any  AMHS,  of 
whatever  capability,  should  be  conducted.  A  cost-benefit  analysis  should  be 
undertaken  to  compare  the  costs  and  benefits  of  the  proposed  AMHS  with  those 
of  the  manual  message-handling  system. 

K.  None  of  the  desirable  features  which  we  recognize  in  the  MME  system  are  of 
such  overriding  operational  significance  that  an  accelerated  effort  is 
necessary  to  field  an  AMHS.  The  development  of  an  optimum  operational 
configuration,  as  recommended  in  paragraph  A  above,  should  continue  to  be  the 
result  of  deliberate,  comprehensive,  user-oriented  efforts. 
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APPENDIX  B.  SYSTEM  USE  DATA 

Table  1  gives  the  total  number  of  users  on  the  system  as  well  as  the  new 

users  introduced  each  week  from  .16  Oct  to  7  Apr.  It  also  defines  the 

periods.  Table  2  gives  the  system  use  by  total  user  hours,  total 

instructions,  and  instructions  per  hour.  Table  3  shows  the  total  number  of 

user  jobs,  the  number  of  jobs  abnormally  terminated,  and  the  corresponding 
percentage  of  jobs  abnormally  terminated.  Table  4  gives  the  total  number  of 
messages  delivered  to  MME  and  the  total  number  of  J3  messages  identified  with 
J3  as  Action  or  Cognizant  addressee.  Table  5  gives  the  number  of  J3  outgoing 
messages  by  day  from  22  February  to  7  April  as  a  percentage  of  all  on-line 
traffic.  Table  6  provides  details  on  all  "Flagged  Instructions"  occurring 
during  the  period  of  this  report.  Table  7  provides  the  system  availability 
for  the  period  covered  by  the  report.  Table  8  presents  a  summary  of  data 
describing  the  use  of  the  MME  in  Exercise  Power  Play  79.  Tables  9  and  10 
present  the  level  of  use  during  the  period  -  table  9  in  terms  of  mean  number 
of  user  hours  per  hour  and  mean  number  of  instructions  per  hour  -  table  10  in 
terms  of  mean  time  on-line  per  day  and  mean  number  of  instructions  per  day 
with  the  means  calculated  by  type  and  by  user  within  the  type. 

Table  1.  CHRONOLOGICAL  SEQUENCE  OF  PERIODS  FOR  THE  REPORT  ALONG  WITH  TOTAL  NUM 
OF  USERS  FOR  EACH  PERIOD  AND  NUM  OF  NEW  USERS  FOR  EACH  PERIOD. 

TOTAL  NUMBER 

PERIOD  DATES  ENCOMPASSED _  OF  USERS _ NEW  USERS 


1 

16  -  18  OCT 

14 

0 

2 

19  -  25  OCT 

26 

0 

3 

26  OCT  -  1  NOV 

30 

2 

4 

2  -  8  NOV 

29 

0 

5 

9  -  15  NOV 

39 

5 

6 

16  -  22  NOV 

50 

8 

7 

23  -  29  NOV 

50 

3 

8 

30  NOV  -  6  DEC 

53 

2 

9 

7-13  DEC 

47 

3 

10 

14  -  20  DEC 

41 

0 

11 

21  -  27  DEC 

47 

6 

12 

28  DEC  -  3  JAN 

49 

4 

13 

4  -  10  JAN 

62 

10 

14 

11  -  13  JAN 

42 

4 

15 

14  -  20  JAN 

62 

6 

16 

21  -  27  JAN 

67 

2 

17 

28  JAN  -  3  FEB 

70 

5 

18 

4-10  FEB 

70 

2 

19 

11  -  17  FEB 

66 

1 

20 

18  -  24  FEB  (FEU) 

57 

3 

21 

25  FEB  -  3  MAR 

70 

0 

22 

4-10  MAR 

80 

8 

23 

11  -  17  MAR 

89 

4 

24 

18  -  24  MAR 

74 

0 

25 

25  -  31  MAR 

71 

1 

26 

1  -  7  APR 

70 

1 

Note:  Full 

Experimental  Use  began  in  period  20. 

The  exercise  was 

conducted 

in  periods  22-24. 
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TABLE  2.  SYSTEM  USE 


PERIOD 

TOTAL  USER 

HOURS 

TOTAL 

INSTRUCTIONS 

INSTRUCTIONS 
PER  HOUR 

I 

31 

350 

11.5 

2 

229 

6685 

29.2 

3 

258 

5078 

19.6 

4 

383 

5576 

14.6 

5 

387 

7789 

20.1 

6 

420 

10205 

24.3 

7 

450 

7889 

17.5 

8 

438 

11191 

25.6 

9 

430 

10366 

24.1 

10 

277 

6707 

24.2 

11 

313 

8564 

27.4 

12 

369 

8799 

23.8 

13 

451 

12911 

28.6 

14 

231 

7416 

32.1 

15 

472 

13957 

29.6 

16 

557 

15270 

27.4 

17 

485 

13429 

27.7 

18 

687 

15322 

22.3 

19 

336 

7429 

22.1 

20 

401 

9191 

22.9 

21 

466 

10757 

23.1 

22 

906 

31841 

35.2 

23 

903 

25421 

28.2 

24 

787 

19888 

25.3 

25 

854 

15316 

17.9 

26 

788 

14423 

18.3 

TABLE  3. 

PERIOD 

USER  JOBS 

TOTAL  NUMBER 

OF  USER  JOBS 

TOTAL  NUMBER  OF 

JOBS  ABNORMALLY 
TERMINATED 

PERCENT  OF  JOBS 
ABNORMALLY 
TERMINATED 

1 

37 

11 

29.7 

2 

143 

38 

26.6 

3 

134 

18 

13.4 

4 

174 

24 

13.8 

5 

246 

26 

10.6 

6 

373 

51 

13.7 

7 

272 

32 

11.8 

8 

398 

60 

15.1 

9 

273 

32 

11.7 

10 

178 

45 

25.3 

11 

262 

76 

29.0 

12 

258 

47 

18.2 

13 

426 

119 

27.9 
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PERIOD 

TOTAL  NUMBER 

OF  USER  JOBS 

TOTAL  NUMBER  OF 

JOBS  ABNORMALLY 
TERMINATED 

PERCENT  OF  JOBS 
ABNORMALLY 
TERMINATED 

14 

246 

62 

25.2 

15 

363 

101 

27.8 

16 

403 

81 

20.1 

17 

373 

72 

19.3 

18 

439 

73 

16.6 

19 

225 

47 

20.9 

20 

239 

33 

13.8 

21 

280 

55 

19.6 

22 

633 

90 

14.2 

23 

655 

77 

11.8 

24 

547 

97 

17.7 

25 

436 

51 

11.7 

26 

404 

47 

11.6 

TABLE  4.  SUMMARY  OF  J3  INCOMING  TRAFFIC 


PERIOD 

TOTAL  NUMBER  OF  MESSAGES 
DELIVERED  TO  MME 

TOTAL  NUMBER  OF  MESSAGES 
J3  ACT/COG 

1 

1331 

620 

2 

3016 

1429 

3 

2981 

1327 

4 

2893 

1304 

5 

2483 

1200 

6 

3066 

1595 

7 

2335 

1243 

8 

3845 

1450 

9 

4678 

1368 

10 

4793 

1640 

11 

3932 

1137 

12 

3442 

1024 

13 

4617 

1214 

14 

2261 

570 

15 

4841 

1321 

16 

4891 

1359 

17 

5053 

1533 

18 

4728 

1316 

19 

5434 

1804 

20 

5271 

1556 

21 

5632 

1576 

22 

7428 

3065 

23 

8039 

3602 

24 

5956 

2029 

25 

5194 

1446 

26 

5457 

1509 

45 


TABLE  5.  SUMMARY  OF  OUTGOING  J3  TRAFFIC  FOR  24  FEB  -  7  APR  (PERIODS  21-26) 


DATE/ 

DAY 

TOTAL  J3 
OUTGOING 

TOTAL  # 
RELEASED 

ON  SYSTEM 

X  OF  TOTAL 
RELEASED 

ON  SYSTEM 

READDRESSALS 

PERIOD  21 

(PLUS  24  FEB) 

FEB 

24 

22 

2 

9.1 

1 

25 

12 

0 

0 

0 

26 

3 

0 

0 

0 

27 

8 

0 

0 

0 

28 

n 

0 

0 

0 

MARCH 

1 

28 

1 

3.6 

0 

2 

24 

4 

16.7 

0 

3 

15 

1_ 

6.7 

0 

TOTAL 

123 

8 

6.5 

'  -l. 

PERIOD  22 

MARCH 

4 

3 

0 

0 

0 

5 

11 

1 

9.1 

0 

6 

58 

24 

41.2 

21 

7 

34 

8 

23.5 

4 

8 

44 

15 

34.1 

15 

9 

45 

10 

22.2 

4 

10 

47 

_9 

19.1 

_5 

TOTAL 

242 

67 

27.7 

49 

PERIOD  23 

MARCH 

11 

32 

4 

12.5 

2 

12 

22 

6 

27.3 

5 

13 

49 

10 

20.4 

6 

14 

34 

11 

32.4 

5 

15 

29 

2 

6.9 

1 

16 

37 

7 

18.9 

7 

17 

53 

14 

26.4 

10 

TOTAL 

222 

54 

24.3 

36 

46 


DATE/ 

DAY 


TOTAL  J3 
OUTGOING 


TOTAL  # 
RELEASED 
ON  SYSTEM 


Z  OF  TOTAL 
RELEASED 
ON  SYSTEM 


READDRESSALS 


PERIOD  24 

MARCH 

18 

24 

7 

29.2 

3 

19 

21 

4 

19.0 

0 

20 

20 

7 

35.0 

6 

21 

22 

4 

18.2 

0 

22 

37 

4 

12.5 

1 

23 

21 

4 

19.0 

2 

24 

23 

_4 

17.4 

3 

TOTAL 

PERIOD  25 

163 

34 

20.9 

15 

MARCH 

25 

3 

0 

0 

0 

26 

10 

3 

30.0 

2 

27 

8 

4 

50.0 

1 

28 

13 

4 

30.8 

1 

29 

14 

8 

57.1 

4 

30 

18 

11 

61.1 

2 

31 

20 

_3 

15.0 

0 

TOTAL 

MARCH 

86 

33 

38.2 

10 

TOTAL 

780 

194 

24.9 

no 

PERIOD  26 

APRIL 

1 

9 

0 

0.0 

0 

2 

10 

8 

80.0 

5 

3 

18 

3 

16.7 

2 

4 

16 

6 

37.5 

4 

5 

12 

3 

25.0 

3 

6 

22 

6 

27.3 

5 

7 

27 

6 

22.2 

4 

TOTAL 

114 

32 

28.1 

23 

TABLE  6.  SUMMARY  OF  FLAGGED  INSTRUCTIONS 


TABLE  7 


INSTRUCTION 

display  message 
display  file 
view  message 
print 

view  directory 

find  entry 

clear  view  window 

save/update 

finish 

file/move 

delete  message 

restrict/augment 

backup 

action  message 
forward  message 
route  message 
reply  message 
display  text 
comment  message 
go  to 

coordinate  message 
release  message 
chop  message 
create  message 
create  file 
create  text 
create  selector 
view  selector 
copy /move/pickup/ put 
keyword  message 
copy  entry 
find  string 
alert 

readdressal 

other 

SYSTEM  AVAILABILITY 


PERIOD 

1 

2 

3 

4 

5 


X  OF  TOTAL  FLAGS 

16 

16 

7 

2 

1 

3 

3 

1 

2 

3 

6 

20 

0.2 

0 

1 

1 

0.1 

1 

0.4 

0.2 

2 

1 

1 

0 

0.2 

0.01 

1 

0.01 

text  2 

0 
1 
2 
0 
1 
3 


PERCENT 

AVAILABILITY  OF 
THE  SYSTEM  TO  USERS 

81.3 

92.4 
96.7 

97.3 

94.3 
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PERCENT 

AVAILABILITY  OF 

PERIOD  THE  SYSTEM  TO  USERS 


6 

89.4 

7 

93.9 

8 

90.5 

9 

94.8 

10 

55.1 

11 

88.1 

12 

78.7 

13 

81.2 

14 

86.3 

15 

87.1 

16 

89.6 

17 

82.0 

18 

90.9 

19 

49.1 

20 

54.8 

21 

66.7 

22 

90.9 

23 

87.1 

24 

88.7 

25 

93.6 

26 

78.0 

TABLE  8.  SYSTEM  USE  AND  DAILY  AVAILABILITY  (EXERCISE  POWER  PLAY  79) 


INCOMING  SYSTEM 
DATE  TRAFFIC  AVAILABILITY 


XO 

ALL  OTHER  PLAYERS 

ACTIVITY 

ACTIVITY 

HRS.  INSTRS. 

HRS.  INSTRS. 

MARCH 


6 

111 

100% 

21:30 

1331 

31:27 

2422 

7 

166 

78% 

14:26 

1031 

21:34 

1803 

8 

182 

100% 

22:00 

1224 

33:20 

2432 

9 

197 

79% 

16:30 

605 

13:38 

895 

10 

197 

95% 

7:30 

699 

28:10 

2200 

11 

192 

100% 

10:00 

484 

24:19 

1350 

12 

168 

72% 

7:30 

702 

22:18 

854 

13 

203 

100% 

13:00 

748 

18:17 

978 

14 

246 

67% 

6:42 

406 

5:28 

356 

15 

163 

100% 

19:08 

368 

12:20 

1391 

16 

137 

100% 

20:00 

870 

14:09 

778 

17 

241 

68% 

16:06 

891 

5:59 

515 

18 

160 

91% 

20:00 

646 

18:26 

863 

19 

124 

82% 

9:30 

815 

14:57 

1164 

20 

105 

94% 

7:44 

308 

22:56 

857 

21 

no 

67% 

2:31 

63 

18:07 

330 

22 

99 

89% 

2:00 

151 

17:39 

806 

23 

10 

98% 

:24 

20 
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TABLE  9  LEVEL  OF  USE 


NOTE: 

Level  of  use  is 

noted  by  period. 

MUH  is  the  mean 

number  of  user 

■  hours 

per  hour,  >LvI 

is  the  mean  number 

of  instructions 

per  hour. 

Level 

of  Use  -  10/19  - 

11/1  1978 

Level  of 

Use  -  11/2  - 

11/15  1978 

HOUR 

MUH 

M#I 

HOUR 

MUH 

MJI 

0000 

.61 

32.57 

0000 

1.39 

20.93 

0100 

.45 

15.14 

0100 

1.48 

11.36 

0200 

1.13 

51.00 

0200 

1.57 

17.50 

0300 

1.26 

42.15 

0300 

1.62 

19.14 

0400 

1.20 

32.79 

0400 

1.74 

29.50 

0500 

1.05 

23.93 

0500 

1.23 

31.50 

0600 

0.46 

14.36 

0600 

1.04 

13.93 

0700 

2.32 

57.60 

0700 

3.89 

132.50 

0800 

3.65 

37.29 

0800 

4.97 

125.43 

0900 

3. SO 

97.64 

0900 

5.20 

127.08 

1000 

3.50 

93.29 

1000 

4.94 

73.64 

1100 

2.27 

39.15 

1100 

4. 74 

59.86 

1200 

1.76 

19.50 

1200 

4.15 

45.43 

1300 

2.69 

50.73 

1300 

4.10 

53.86 

1400 

3.00 

65.79 

1400 

3.87 

77.66 

1500 

2.35 

67.11 

1500 

2.99 

75.27 

1600 

1.36 

25.90 

1600 

1.17 

28.34 

1700 

1.14 

15.43 

1700 

.81 

18.00 

1300 

.50 

9.18 

1800 

.70 

27.25 

1900 

.56 

10.60 

1900 

.98 

18.75 

2000 

.70 

24.22 

2000 

1.46 

15.90 

2100 

.70 

73.86 

2100 

1.37 

21.18 

2200 

.90 

14.32 

2200 

1.33 

20.08 

2300 

1.01 

14.47 

2300 

1.52 

17.32 

! 

» 

i 

i 

50 

> 

*  ■ 

L 

— 

J 

Level  of  Use  -  11/16  -  11/29  1978 


HOUR 

MUH 

Mi?  I 

0000 

1.27 

15.36 

0100 

1.  31 

13.93 

0200 

1.34 

19.79 

0300 

1.35 

11.07 

0400 

1.40 

21.28 

0500 

1.08 

16.50 

0600 

1.27 

27.21 

0700 

4.61 

138.86 

0800 

5.45 

152.64 

0900 

5.35 

126.21 

1000 

5.04 

85.57 

1100 

4.88 

74.71 

1200 

4.79 

90.65 

1300 

4.92 

122.57 

1400 

2.36 

83.85 

1500 

3.00 

68.07 

1600 

1.71 

29. 72 

1700 

1.21 

27.23 

1300 

.66 

10.24 

1900 

.86 

7.37 

2000 

.96 

14.54 

2100 

1.08 

19.52 

2200 

1.14 

14.07 

2300 

1.27 

16.00 

Level  of  Use  -  11/30  -  12/13  1978 


HOUR 

MUH 

M//I 

0000 

1.26 

31.50 

0100 

1.69 

52.50 

0200 

1.50 

22.43 

0300 

1.51 

40.22 

0400 

1.75 

43.72 

0500 

1.49 

27.57 

0600 

1.48 

38.86 

0700 

5.17 

140.21 

0800 

5.20 

136.86 

0900 

6.54 

132.11 

1000 

5.87 

149.50 

1100 

5.10 

90.00 

1200 

4.43 

93.36 

1300 

4.98 

155.64 

1400 

4.59 

135.42 

1500 

4.93 

134.33 

1600 

2.52 

36.17 

1700 

2.29 

37.17 

1800 

.  74 

13.50 

1900 

.83 

14.  75 

2000 

1.01 

21.50 

2100 

.88 

33.75 

2200 

1.34 

26.25 

2300 

1.43 

27.07 

MUH  -  mean  number  of  user-hours  per  hour 
M;/ 1  -  mean  number  of  instructions  per  hour 
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Level  of  Use  -  12/14  -  12/27  197S 


HOUR 

MUH 

Mi'/ 1 

0000 

1.52 

42.67 

0100 

1.24 

18.85 

0200 

1.48 

21.27 

0  300 

1.57 

31.39 

0400 

1.64 

36.24 

0500 

1.32 

24.67 

0600 

1.17 

33.80 

0700 

3.70 

113.21 

0800 

4.70 

103.27 

0900 

4.  78 

116.10 

1000 

4.47 

106.14 

1100 

4.26 

78.11 

1200 

3.57 

78.23 

1300 

3.60 

133.00 

1400 

2.99 

69.10 

1500 

3. 22 

61.75 

1600 

2.07 

19.00 

1700 

1.16 

32.42 

1800 

.93 

17.50 

1900 

.86 

20.58 

2000 

1.15 

28.75 

2100 

1.12 

31.38 

2200 

1.03 

21.37 

2300 

1.29 

23.30 

Level  of  Use  -  12/28/78  -  1/10/79 


HOUR 

MUH 

Mi/I 

0000 

1.39 

21.43 

0100 

1.44 

21.07 

0200 

1.92 

23.57 

0300 

1.74 

3S.79 

0400 

1.65 

38.57 

0500 

1.45 

32.79 

0600 

1.70 

59.14 

0700 

4.48 

162.00 

0800 

5.22 

161.14 

0900 

5.59 

135.36 

1000 

6.36 

158.00 

1100 

5.71 

122.50 

1200 

5.15 

113.36 

1300 

5.49 

143.29 

1400 

4.  t>4 

134.00 

1500 

3.67 

32.17 

1600 

2.07 

25.33 

1700 

1.85 

26.33 

1800 

.60 

5. SO 

1900 

.98 

53.58 

2000 

.  S2 

48.  75 

2100 

.89 

35.82 

2200 

1.14 

32.21 

2300 

1.35 

64.14 

XUH  -  mean  number  of  user-hours  per  hour 
M:‘I  -  mean  number  of  instructions  per  hour 
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Level  of  Use  -  2/22  -  3/ 10  1979 


Level  of  Use  -  3/ 1 1  -  3/24  1979 


HOUR 

MUH 

MI 

coon 

2.99 

65.00 

0100 

2.92 

66 . 40 

0200 

2.84 

38.40 

0300 

3.27 

122.07 

0400 

3-27 

109.53 

0500 

3.22 

113.47 

0500 

3.91 

176.13 

0700 

7.77 

302.13 

0300 

3.43 

231 .67 

0900 

8.55 

207.53 

1000 

8.53 

214.29 

1100 

7.14 

158.86 

1200 

6.41 

107.00 

1300 

7.27 

163.08 

1400 

8.12 

163.00 

1500 

5.49 

212.75 

1600 

4.25 

133.08 

1700 

3-38 

95.90 

1800 

2.23 

56.08 

1900 

3.04 

63.64 

2000 

2.74 

93.14 

2100 

2.77 

99.47 

2200 

2.91 

66.13 

2300 

2.52 

78.07 

MUH  =  mean  number  of  user-hours  per  hour 
M#I  =  mean  number  of  instructions  per  hour 


HOUR 

MUH 

0000 

.  3-70 

0100 

3.98 

0200 

4.09 

0300 

3.91 

0400 

4.07 

0500 

3-76 

0600 

4.32 

0700 

7.67 

0800 

8.56 

0900 

9.08 

1000 

9.02 

1100 

8.22 

1200 

7.47 

1300 

8.05 

1400 

8.23 

1500 

8.69 

1600 

5.24 

1700 

4.17 

1800 

3.46 

1900 

3.69 

2000 

3.82 

2100 

4.04 

2200 

4.05 

2300 

3.69 

Level 

of  Use 

HOUR 

MUH 

0000 

2.60 

0100 

2.78 

0200 

2.83 

0300 

2.94 

0400 

3-32 

0500 

3.23 

0600 

3.45 

0700 

9.08 

0800 

10.32 

0900 

10.64 

1000 

10.70 

1 100 

9.87 

1200 

9.78 

1300 

10.66 

1  400 

11.24 

1500 

9.98 

1600 

5.09 

r’oo 

2.54 

1  SCO 

2.07 

1900 

2.38 

2000 

2.56 

2100 

2.56 

2200 

2.53 

23CC 

2.«2 

Ml 

82.54 

77.29 
53.21 

55.79 

59.29 

95.29 
174.57 
304.43 
225.14 
211.33 

224.36 

179.50 

126.50 
218.90 

264.36 
245.20 
133.80 
104.00 

93.80 

114.36 

71.36 

101 .50 
94.42 
85.69 


3/25  -  4/7  1979 
Ml 

34.92 

41.92 
32.75 
30.69 

46.23 

89.30 
151  .84 
233.46 
192.54 
204.00 
184.60 
136.76 

35.23 
182.15 
167.09 
1 34 .  -6 

86.00 

34.40 

26.65 

37.39 

32.80 

49.31 

19.92 
43.62 


53 


TABLE  10  '  LEVEL  OF  USE  BY  USER  TYPE 


Note 


This  table  presents  the  average  use  by  users  within  a  particular  type 
(the  types  are  defined  in  Section  3).  The  mean  time  on-line  per  day 
is  listed  for  each  type  of  user  and  for  the  users  within  each  type. 
The  mean  number  of  instructions  per  hour  is  also  listed  by  user-type 
and  by  user  within  type. 


LEVEL  OF  USE  BY  USER  TYPE  -  16  Oct  -  1  Nov  1978 


TYPE 

MEAN 

PER  TYPE 

MEAN 

PER 

USER 

Time- 

it  Inst 

Time 

it  : 

(min. 

) 

(min. 

) 

DBADM 

165 

18 

165 

18 

J301 

149 

85 

119 

68 

AO/ c. 1 

268 

S2 

189 

58 

AO/c.2 

169 

92 

66 

36 

AC/ c. 3 

132 

67 

113 

58 

DBClr 

110 

17 

88 

14 

CCAIR 

495 

33 

377 

25 

CCS  UR 

344 

90 

206 

56 

DDO 

4 

12 

4 

12 

JRC 

660 

304 

373 

56 
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LEVEL  OF  USE  3Y  TYPE  -  2  Nov  -  15  Nov  1978 


TYPE 

MAN  PER  TYPE 

Time  #  Inst 

(min. ) 

MEAN  PER 
Time 
(min. ) 

USER  IN  TYPE 
it  Inst 

DBADM 

546 

61 

238 

27 

J301 

270 

180 

202 

120 

AO/c.l 

6  38 

104 

355 

58 

AO/c.2 

245 

170 

60 

40 

AO/c.3 

333 

129 

141 

57 

DBClr 

199 

28 

150 

22 

CCAIR 

864 

119 

546 

80 

CCS  UR 

632 

175 

374 

109 

DDO 

55 

14 

55 

14 

JRC 

488 

107 

285 

47 

LEVEL  OF  USE 

BY  TYPE  - 

16  Nov  -  29 

Nov  1978 

TYPE 

MEAN  PER  TYPE 

Time  it  Inst 

(min. ) 

MEAN  PER 
Time 
(min. ) 

USER  IN  TYPE 
/rlns  t 

DBADM 

548 

74 

214 

30 

J301 

258 

210 

166 

135 

AO/c.l 

612 

81 

408 

51 

AO/c.2 

228 

227 

40 

38 

AO/c. 3 

319 

168 

74 

42 

DBClr 

25 

76 

16 

CCAIR 

615 

i6 

410 

29 

CCSUR 

541 

160 

306 

95 

DDO 

73 

~  / 

73 

^  T 

JRC 

594 

196 

223 

84 

55 
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LEVEL  OF  USE  3Y  TYPE  -  30  Nov  -  13  Dec  1978 


TYPE 

MEAN  PER 
Time 
(min. ) 

TYPE 
if  Inst 

MEAN  PER 
Time 
(min. ) 

USER  IN  ' 
if  Inst 

DBADM 

611 

108 

272 

54 

J301 

278 

236 

167 

142 

AO/c.l 

396 

55 

207 

28 

AO/c.2 

253 

231 

48 

49 

AO/ c. 3 

132 

106 

40 

32 

DBClr 

190 

48 

105 

26 

CCAIR 

571 

135 

320 

76 

CCS  UR 

521 

142 

261 

142 

DDO 

47 

32 

47 

32 

JRC 

884 

272 

387 

98 

LEVEL  OF  USE 

BY  TYPE 

-  14  Dec  - 

27  Dec  1978 

TYPE 

MEAN  PER 
Time 
(min . ) 

TYPE 
if  Inst 

MEAN  PER 
Time 
(min . ) 

USER  IN 
if  Inst 

DBADM 

520 

143 

260 

71 

J301 

311 

193 

173 

107 

AO/c.l 

376 

109 

150 

44 

AO/c.2 

108 

128 

36 

43 

AO/c.3 

315 

152 

76 

40 

DBClr 

242 

68 

161 

45 

CCAIR 

171 

34 

133 

26 

CCS  UR 

539 

109 

314 

63 

DDO 

25 

45 

25 

45 

JRC 

944 

355 

340 

128 
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LEVEL  OF  USE  BY  TYPE  -  28  Dec  1978  -  10  Jan  1979 


TYPE 

MEAN  PER  TYPE 

Time  i?  Inst 

(min. ) 

MAN  PER 

Time 

(min.) 

USER  IN  TYPE 
#  Inst 

DBADM 

677 

107 

212 

33 

J301 

362 

246 

155 

105 

AO/ c.  1 

552 

184 

199 

64 

AO/c.2 

318 

336 

49 

54 

AO/c.3 

378 

265 

86 

59 

DBClr 

351 

76 

131 

31 

CCAIR 

378 

92 

189 

46 

CCSUR 

495 

67 

292 

41 

DDO 

29 

33 

23 

26 

JRC 

701 

206 

256 

75 

57 


LEVEL  OF  USE 

BY  USER 

TYPE  -  22  Feb  - 

10  March 

1979 

TYPE 

MEAN 

PER  TYPE 

MEAN 

PER  USER  IN  TYPE 

Time 

it  Inst 

Time 

it  Inst 

(mm 

.) 

(min . 

) 

D3ADM 

697 

128 

246 

45 

J301 

388 

244 

187 

117 

AO/ c . 1 

490 

117 

212 

51 

AO/c.2 

349 

310 

74 

65 

AO/c.3 

348 

183 

75 

40 

DBClr 

195 

76 

74 

29 

CCAIR 

368 

252 

397 

115 

CCSUR 

590 

181 

359 

1 10 

DDO 

376 

29 

301 

23 

JRC 

870 

203 

398 

93 

DBMgt 

281 

71 

121 

29 

EXERCISE** 

Non-J3 

165 

226 

71 

97 

AOs 

865 

939 

242 

263 

CLR 

54 

84 

54 

84 

SUR 

140 

88 

140 

88 

MGT 

339 

102 

242 

87 

**  Non-J3  are  people  who  use  Sigma  for  exercise  only.  Others  are 
users  in  J3  who  were  logged  on  in  exercise  accounts  (J3x,  etc.) 
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LEVEL  OF  SIGMA 

USE  BY 

USER  TYPE  - 

1 1  March  -  24 

March  1979 

TYPE 

MEAN 
Time 
{ min . 

PER  TYPE 
ft  Inst 

) 

MEAN  PER 
Time 
(min .) 

:  USER  IN  TYPE 
it  Inst 

DBA  DM 

594 

110 

148 

28 

J301 

324 

187 

139 

81 

AO/c.1 

564 

145 

211 

46 

AO/c.2 

249 

302 

51 

52 

AO/c.3 

255 

157 

56 

34 

DBClr 

285 

58 

95 

20 

CCAIR 

1007 

254 

455 

115 

CCSUR 

478 

167 

261 

91 

CCDDO 

104 

23 

93 

20 

JRC 

11  13 

227 

410 

84 

DBMgt 

727 

196 

200 

54 

EXERCISE** 

fIon-j3 

590 

511 

114 

99 

J301 

442 

48 

442 

48 

AOs 

9333 

585 

250 

156 

Clr 

203 

126 

203 

126 

S'jr 

32 

73 

32 

73 

**  Mon-J3  are 

people 

who  use  Sigma 

for  exercise 

only.  Otners 

users  m  J3  wno  were 

logged  on  in 

exercise  accounts  (J3x,  etc 
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LEVEL  CF  SIGMA  USE  BY  USER  TYPE  -  24  March  -  7  April  1979 


TYPE 

MEAN 

Time 

(mm 

PER  TYPE 
ft  Inst 

.) 

MEAN 
Time 
(min , 

PER  USER  IN  TYPE 
ft  Inst 

.) 

DBADM 

1107 

194 

252 

44 

J  30 1 

535 

319 

153 

85 

AO/c.1 

308 

161 

342 

68 

AO/c.2 

655 

310 

109 

52 

AO/c.3 

655 

296 

124 

56 

DBClr 

239 

69 

133 

38 

CCAIR 

1177 

328 

464 

129 

CCSUR 

744 

189 

345 

88 

CCDDO 

293 

18 

183 

11 

JRC 

1415 

209 

483 

71 

DBMgt 

513 

113 

138 

30 
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